Re: Best queue system for myrinet?

> 
> Actually, we have a similar config here (with the Maui scheduler for
> node selection) on five 32+ dual node clusters and in my experience
> OpenPBS needs quite a bit of hand holding.  Having to restart PBS
> services often for various (usually unclear) reasons is in fact quite
> a pain.  I must immediately add that we don't have ALL the patches of
> the above site, but as far as I know we do use the relevant ones.
> 
> Maybe other people can comment on SGE (Sun Grid Engine, formerly Codine).
> It seems to offer a similar interface, is now open software, and
> possibly a bit more stable?

We had been using OpenPBS on the clusters here at FSL (256 nodes) since
the start of the contract..  We had some patches from an previous employee and I
tracked the work of others to try and improve the stability of OpenPBS.  Nothing
really worked.  The servers would get hung quiet often when nodes would
hang.  I really like the feature set of OpenPBS, but the comm model
(server talks to clients) isn't good, and didn't scale.

We decided to try other queueing systems.  We started with SGE because
it was free.  It feels much more like a server farm batch system
than one designed for multi-node jobs.  However, there are facilities
for running mpi jobs.    It is very stable.  I have had SGE running
with systems of 150 and 768 nodes, and the servers has never crashed
execpt for hardware or kernel panics (The kernel panics are not SGE related).
SGE has the ability to have shadow servers so the users don't even know execpt for 
a 2 minute hiccup.

Maybe there are some better patches out their to fix OpenPBS.   I haven't looked
at PBS in the last 6 months. However, I found SGE to be very stable and I just put a 
little work into making it behave as I wanted.  


Craig


Craig Tierney ([email protected])