Search My Blog

Tuesday, September 29, 2009

PelicanHPC: A GNU/Linux distribution to create a HPC cluster for MPI based parallel computing

Features

PelicanHPC is a live CD image that let's you set up a high performance computing cluster in a few minutes. A Pelican cluster allows you to do parallel computing using MPI. You can run Pelican on a single multiple core machine to use all cores to solve a problem, or you can network multiple computers together to make a cluster. The frontend node (either a real computer or a virtual machine) boots from the CD image. The compute nodes boot by PXE, using the frontend node as the server. All of the nodes of the cluster get their filesystems from the same CD image, so it is guaranteed that all nodes run the the same software. Packages can be added to the frontend node on the fly, thanks to aufs. If new software is installed in /home/user (easy for self-compiled, a little tricky for .debs), it is available to all the nodes. The CD image is created by running a single script, which takes advantage of the Debian Live infrastructure.  It is very easy to create a custom version with new packages installed in standard locations by adding package names to the script and then running it.
  • Pelican is created using Debian Live. To make your own version you only need live_helper (also deboostrap, rsync, any others?) and the make_pelican script, which is provided below.
  • The LAM-MPI and OpenMPI implementations of MPI are installed. Both 32 and 64 bit versions are available. Debian testing (Lenny) is the base for both.
  • Contains extensive example programs using GNU Octave and MPITB. Also has the Linpack HPL benchmark.
  • You can use any Class C network you like. By default, the cluster is on 10.11.12.*
  • xfce4 window manager, konqueror for browsing and file management, ksysguard for monitoring the cluster, kate and nano for editing. As noted, it is very easy to add packages. Pelican is a bare-bones framework for setting up a HPC cluster.
  • Pelican releases and all testing is currently done using Debian Lenny as the base. Squeeze or sid may or may not work.
I have looked it over in VirtualBox and it looks and sounds great! Now to set up that Cluster!:)

Read more...
http://idea.uab.es/mcreel/ParallelKnoppix/

Don

No comments: