Re: [myrinet] re: port held open and killing processes

From the man page for rsh,

   Job control signals stop the local rsh process only; this is 
   arguably wrong, but currently hard to fix.

Under linux TERM signals are sent through the rsh but none others. The
problem is that you are not allowed to have signal handling for KILL.
Makes sense. So rsh can't catch the signal and send it to the remote
process and then exit.

This is what I was trying to get at the other day, we need to get better
signal handling in the mpich device so we're not relying on rsh and a
bunch of shell scripts. I posted a patch that could be used beginnings of
of a fix for this this problem.

Doug

On Thu, 21 Sep 2000, Bob Felderman wrote:

> => I was able to get things in a state where I killed off the 
> => rsh commands on the local node, but the remote node
> => still had the gm_allsize process holding open the port.
> => This is exactly what we have seen with the mpirun 
> => process not being able to kill off the remote mpi job.
> => 
> => I'm trying a bit more investigation.
> 
> 
> It appears that a 
>     kill -9 <pid>
> of the rsh on the local node leaves the remote job still running.
> Using
>     kill <pid>
> on the local node seems to kill off the remote process cleanly.
> 
> Does that make sense to anyone?
> 
> It looks to me like our stock mpirun.ch_gm
> uses kill 'KILL' which is just kill -9
> 
> frisbee.myri.com 31% grep kill mpirun.ch_gm.in 
>    print STDERR " --gm-kill <n>  n secs after first process exits, kill all other processes \n";
>      if (kill 0 => $p) {
>        print "killing process $p\n" if $verbose;
>        kill 'KILL', $p;
>     } elsif ($_ eq '--gm-kill') {
>         print "option kill procs after $ARGV[0] seconds\n" if $verbose;
>         $kill_after_t = 1;
>         printf("kill processor %d seconds after first exits\n",$time) if $verbose;
> if ($kill_after_t){
>     if (kill 0 => $p){
>       print "1 process still alive: wait $time sec, then kill it\n" if $verbose;
>       print "$still_alive processes still alive: wait $time sec, then kill them\n" if $verbose;
>       if (kill 0 => $p) {
>         print "killing process $p\n" if $verbose;
>         kill 'KILL', $p;
> 
> 
> 

Doug Johnson
Systems Developer/Engineer
The Ohio Supercomputer Center
[email protected]