Re: [myrinet] RE: mpich PBS etc. and gm_ports not closing

Hi,

I'm not using PBS, however I am seeing problems with mpi processes left
running after a non-clean exit by one of the nodes. I have never seen a
port open without an associated process in existence. Typically they are
using a high percentage of cpu, which suggests they are sitting in a spin
loop. 

I use an awk script to parse gm_board_info after
job has finished, however I am having intermittent problems with
gm_board_info hanging, in which case I have to go round killing processes
(both gm_board_info and mpi processes).

Best wishes,

Peter

On Tue, 19 Sep 2000, Greg Lindahl wrote:

> > I'd like to get some idea of how wide-spread this problem is.
> > We hear from customer A that they have talked with customer
> > B,C,D and all of them do something like reloading GM
> > between user's jobs to be sure the ports are cleaned up.
> 
> I *used* to do that, back with gm-1.0.X. It has not been necessary since,
> except for an early 1.2.X which was unusually buggy. I suspect most of the
> rumors about that are stale, since people seem to think I'm still doing that
> on the FSL system.
> 
> Now "cleaning up ports" is an ambiguous problem. It could be the case that
> these folks are having a problem getting all their MPI or other
> myrinet-using processes to exit. The next program to come along will then
> complain that the port is in use. In that case, one way to "clean up ports"
> is to reload GM, but I prefer using a script which parses the output of
> gm_board_info and kills the guilty processes.
> 
> Of course, other people can shed more light on their problems...
> 
> -- greg
> 
>