Re: [myrinet] --disable-directcopy in mpich

[email protected] said:
> Pete Wyckoff wrote:
> > 
> > The CVS version of GM will lockup our quad processor x86 linux boxes if
> > directcopy is enabled.  While I'm trying to figure this out, for
> > production we disable the option.
> 
> Hi,
> 
> Do you mean GM works fine without compiling the directcopy code and
> lockup if the code is compiled ? Does the lookup occurs with GM only or
> with MPICH-GM ?
> The directcopy code is in the GM driver because it works at kernel
> level, but GM never uses it directely, only MPICH-GM does.

I must use --disable-directcopy in the GM tree, then my patch and the
same option in the MPICH tree, to get a version which does not lockup.

The test case I have is one of the NPB codes, and only happens when
running multiple processes per node.  I didn't check if 2 is sufficient,
but always put 4 processes on each quad processor box in a 16-processor
job.  A seemingly random one of the nodes will have all the processes
using /dev/gm* move into 'D'isk wait state trying to down a semaphore.

I don't have a pure GM test case, unfortunately, to help locate the
problem.

> > Mpich, though, doesn't detect this and always forces the use of
> > directcopy, making it incompatible with GM.
> Yes, because the directcopy support is stable enough to be the default
> with Linux boxes. The lookup that you notice is not normal and should be
> fixed, maybe off mailing-list via [email protected].

It doesn't even try to lookup in the GM build.  I was just dreaming of a
perfect future by which MPICH learns some of its build options from the
GM build.  :)  Noticed you took out the version lookup too.  Good thing.

> > Ideally mpich configure would be taught to go grepping through the
> > GM build tree to determine if it should do directcopy or not.  For
> > now, would you add this to the tree?
> Yep, no problemo.

		-- Pete