Re: remapping myrinet while jobs are running

On Fri, 2005-06-24 at 17:42 -0400, John C. Wright wrote:
> Unfortunately, this is not our experience.  Rebooting a node kills all
> running jobs.  I thought it should work this way, but it 'hangs' any
> job running on the network when a node (not involved in one of the
> running jobs) is rebooted.  Are there certain mapping settings that
> are needed or to be avoided for this to work as you describe?

Have you checked your network for loose, dirty, or bad cables?  From
past experience I have had a bad or misseated cable cause all sorts of
problems, even causing the mapper to fail to map parts of the network
(the ports still had a link light).

If you havn't debugged your network the I would recommend getting
gm_auto_stress out of one of the newer GM2 source tarballs and using
that as your traffic generator.

I would start with http://www.myri.com/cgi-bin/fom?file=52 for info on
finding bad connections.  There are a bunch of FAQ entries related to
this.

Josh