PelicanHPC GNU Linux
19 June 2013. End of life for PHPC, no further releases will
be made. The download page will be available for a while.
PelicanHPC is an iso-hybrid (CD or USB) image that let's you set up a high performance computing cluster in a few minutes. A Pelican cluster allows you to do parallel computing using MPI. You can run Pelican on a single multiple core machine to use all cores to solve a problem, or you can network multiple computers together to make a cluster. The frontend node (either a real computer or a virtual machine) boots from the image. The compute nodes boot by PXE, using the frontend node as the server. All of the nodes of the cluster get their filesystems from the same image, so it is guaranteed that all nodes run the the same software. Packages can be added to all nodes using apt-get, thanks to aufs. The bootable image is created by running a single script, which takes advantage of the Debian Live infrastructure. It is very easy to create a custom version with new packages by adding package names to the script and then running it. PelicanHPC is creates a fairly bare-bones cluster, with few applications other than those relevant to the author's research and teaching. To use PelicanHPC for serious work in other disciplines, it will be necessary to create a customized version. The tutorial explains how to do this. It is not very difficult. Two examples of specialized distros that add to the PelicanHPC base are MOLA and birgHPC. These are very good examples of how it is intended for the make_pelican script to be used.
Pelican is created using tools from the Debian Live project. To make your own version you only need live-build (also deboostrap and rsync) and the make_pelican script, which is provided below. All of this is also available on the PelicanHPC images, so you can build a custom version while running PelicanHPC.
Contains the latest stable version of the OpenMPI implementation of MPI.
Available for 64 bit CPUs. An older version is available for 32 bit CPUs.
Contains extensive example programs using GNU Octave. Also has the Linpack HPL benchmark.
Pelican releases and all testing is done using the stable (Squeeze) version of Debian GNU Linux as the base.
Here are some screenshots (click to enlarge):
Use the normal download page or a mirror (see below). The make_pelican script and pre-built ISO images are always available on the normal download page. To use the script to make your own version, you need to install the live_helper package. Get the most recent version.
Germany:
http://download.mi.hs-heilbronn.de/pelicanhpc/
Provided by the Medical
Informatics department of Heilbronn University and Heidelberg
University.
http://download.semmel.org/PK/
username "parallel" password "knoppix"
The main documentation is the tutorial, (also in Russian) which is a series of screenshots with comments. For those of you who fly by the seat of your pants, boot a computer using the CD, and follow the instructions that come up on the screen.
An academic paper style
discussion is in the document pelicanhpc.pdf.
If
any
reputable
journal
would
like
to
publish
this,
please
get
in
touch.
Warning: this is somewhat out of date. Screencast video that shows how to set up and use PelicanHPC. Warning - this is about 34MB.
User Forum: the main place to ask questions
How to use ksysguard as a cluster monitor. (Thanks to A. Balsa)
Article in Linux Pro Magazine from June 2009.
Please
let me know if you use this software to produce published academic
work.
Some other distros for setting up a HPC cluster are mentioned here. They may be better choices for certain tasks.
MOLA is a distro for virtual screening of small molecules. It was made by modifying make_pelican. Here's a tutorial that shows how it works.
birgHPC is a distro for bioinformatics built using an adapted version of make_pelican. It adds MPICH2, as well as a suite of tools for bioinformatics including GROMACS.
KestrelHPC is a system for setting up a cluster of netbooted clients, using a frontend machine that runs Debian or Ubuntu GNU/Linux. It is available as a .deb package. Now at its second version.
ABC Gnu/Linux is a distro for setting up a HPC cluster that works in both live and installed modes. I have tried it in live mode on a virtual cluster, and it works nicely, in my experience. It has the ganglia monitor, which is nice for seeing the details of the operation of the cluser. The frontend node automatically begins to operate as a dhcp server, using eth0, so be careful not to interfere with existing networks if you boot it up.
Cluster by
Night is another live CD approach to setting up a
cluster for MPI work. It works within an existing network,
which is not the case with PelicanHPC. It seems to be a very
nice solution when it is not possible to set up a private
network.
This part is here to help convince
people that funding this project is worthwhile. Check
at StatCounter (then click on Recent Visitor Map) to see an
updated version of this map:
(c)
2011 Michael Creel