Re: [myrinet] mpich PBS etc. and gm_ports not closing

I think this has everything to do with how the parallel job was
started. The rsh may be terminated by the C-c but the process on the other
end may not die (leaving the port in use). Is this the problem? Or are
people seeing open ports with no process? 

I think _some_ of these problems will be fixed when MPI_Abort works
properly (next mpich). The other solution is to change how the parallel
program is started. At OSC we've created a c program that uses the task
manager primitives provided by PBS to start the mpi processes on the nodes
that have been allocated for that job. The problem with that is that the
TM layer in PBS is not yet reliable enough to rely on it for high
processor count jobs. This program can be downloaded from,

http://www.osc.edu/~djohnson/mpiexec

We're at version 0.4 now. There's a diff for PBS included in that for
version 2.2p11.

There are lots of other approaches that could be taken to fix this
problem, but none of the shell script versions are particularly elegant.
At the MPICH layer there should be signal handling that catches the C-c
and calls the new and improved MPI_Abort. Should be a pretty simple fix...

Doug


On Tue, 19 Sep 2000, Bob Felderman wrote:

> Some site have reported problems with GM ports not closing
> properly when using PBS and hitting ctrl-C etc.
> 
> I'd like to get some idea of how wide-spread this problem is.
> We hear from customer A that they have talked with customer
> B,C,D and all of them do something like reloading GM 
> between user's jobs to be sure the ports are cleaned up.
> 
> That's clearly not what we expect users to be doing, so if we
> can hear what sort of problems (and solutions) people
> have, we can start to try to fix them.
> 
> Please send email back to this list in case the response is
> of some use to others.
> 
> Thanks,
> Bob
> 
> ------------------------------------------------------------------
> Bob Felderman                                 (626) 821-5555
> Director of Software Development              (626) 821-5316 fax
> Myricom Inc.                                  [email protected]
> 325 N. Santa Anita Ave.                       http://www.myri.com
> Arcadia, CA 91006
> ------------------------------------------------------------------
> 

Doug Johnson
Systems Developer/Engineer
The Ohio Supercomputer Center
[email protected]