RE: mpich-gm1.2.5..10 and PBSpro

"James Lowey" <[email protected]> Wed, 15 Oct 2003 08:32:34 -0700
Newsgroups gmane.network.myrinet.general
Message-ID <[email protected]>
Thanks for the response, 

-----Original Message-----
From: Patrick Geoffray [mailto:[email protected]] 
Sent: Tuesday, October 14, 2003 5:21 PM
To: James Lowey
Cc: [email protected]
Subject: Re: [Myrinet] mpich-gm1.2.5..10 and PBSpro


>You have a rogue process that keeps sending the same init message over
and over to the mpirun.ch_gm.pl script. Can >you add the flag "-v" to
have more info ? Also, could you run tcpdump during the job to see who
is sending these messages ? No MPICH-GM process should send anything
after the MPI job has started. So if you spawn an MPI code that sleeps
after printing something, and you see all of them reaching the sleep,
and you see someone sending to the mpirun.ch_gm.pl script, this is man.

If I submit the job with the -v option and the original un-modified
mpirun.ch_gm.pl script the output is about 
4.5 MB.   This is using the example cpi program.

If I submit the job with the -v option with the modified mpirun.ch_gm.pl
script the output is about 33k.
However it appears to execute normally.. 



>> message that is there.  We came up with a workaround by modifying the

>> mpirun.ch_gm script setting warnings to 0 but when you submit a job 
>> via PBSpro the head mom

>How did you set the warnings to 0 ?

In the mpirun.ch_gm.pl script added a $warnings = 0; 
then added a if $warnings; to the end of the lines with <warn ("Recieved
invalid data format !\n")> and similar lines in the script


> node shows a perl process from mpich-gm consuming a tremendous amount 
> of processor resources. So the "fix" only masks the problem, it does 
> not solve it.

The root cause should be found, this is not normal. BTW, does it work
without PBSpro, ie using mpirun.ch_gm by hand ?

It works fine if you submit the job manually without using PBSPro.