RE: Best queue system for myrinet?

 

-----Original Message-----
From: Thomas Davis
To: Bill Broadley
Cc: [email protected]
Sent: 11/18/02 1:13 PM
Subject: Re: [Myrinet] Best queue system for myrinet?

Bill Broadley wrote:
> I'm reinstalling a 32 node (64 cpu) myrinet cluster.
> 
> I'm looking for a batch queue system, but want to avoid the problem
> I've had in the past (although this info maybe somewhat dated).
>> 
>> Basically I want to schedule my cluster as 64 cpu's, but if I
>> tell the queue system I have more then 32 cpu it needs to know
>> that the first job on a given node gets Myrinet channel 2, and the
> second 
>> job on the same node gets Myrinet channel 4.
>> 
>> What is working well for people?
>> 
>
>We have been doing some batch queue testing with ESP2, becasue we are 
>greatly disappointed with the reliability (and ability of) OpenPBS.
>
>
>We've tested to date the following:
>
>1) OpenPBS w/Maui
>2) SGE (free edition)
>3) Platform's LSF
>4) Maui/ME
>
>The problems of each:
>
>1) Hangs; has tendency to put a job in a wierd state, that sends 
>thousands of email messages saying so; only way to clear is stop the 
>master, and delete the job file by hand.
>
It hangs because the server tries to talk to the nodes actively,
and if the node doesn't respond the server hangs.  Try a 
strace on the server process and see what is happening.

>
>2) Scheduler doesn't work very well - wants to run all the small jobs 
>first, priorities only apply after jobs are sorted by size.
>

I have not seen this with SGE.  I do not have this problem.  I have
seen where the scheduler does not schedule the oldest job though 
(all same size, all same priority).


Craig