RE: [Myrinet] [Myricom help #13378] hybrid mpich support

> On Mon, 2002-10-07 at 19:03, Van Maren, Kevin wrote:
> > Patrick,
> > 
> > My use of "hybrid" was very unfortunate, and I think that may
> > be leading to confusion.
> 
> I see. Indeed, I was referring to hybrid as one of the receive mode.
> Your patch makes much more sense in this context. 
> 
> > run on (beyond the scope of mpirun -- I do it similar to running
> > a program under gdb, with a wrapper program that reads the MPI
> > env variables and does the "right thing"), and you need the machine
> 
> Are you sure that mpirun cannot help you there. mpirun.ch_gm is quite
> open, we can add some specific support if needed.

I wasn't sure what to do there -- the problem is that every OS uses
different ways to bind processors, and you need to "know" which
processors are where, etc.  Since it was just a proof-of-concept
change that I thought might be useful, I left it with a hack:
I added the logic that had to be in there, and left the rest to the
application to deal with.

> > I'm not claming a hybrid approach is right in all (any?) 
> circumstances,
> > but it is something to experiment with, especially if you 
> have a NUMA
> > or multiple-processor-bus machine with a Myrinet interface per node.
> 
> The vast majority of our customers use single or dual CPU 
> boxes, rarely quad, but not NUMA.
> So we have never investigated this case where the
> memory access is not uniform. A such hybrid configuration 
> makes perfect
> sense (it would even make much more sense if there would not 
> be a memory
> copy for small messages in MPICH-GM to deal with registration and GM):
> using the NIC for local communications can be seen as an asynchronous
> offloaded memory copy. In a perfect world, a such hybrid 
> behavior would
> be extended to different message sizes: shared memory for small
> messages, even the cache-coherency overhead would be small compare to
> the gain in latency, but Myrinet for large messages to avoid CPU waste
> and cache trashing. However, I agree that a spacial partitioning is a
> good first step.
> 
> I will look closely at your patch and including it before 1.2.4..8b.

Ok.  If nothing else, it sounds like renaming "hybrid" to something equally
descriptive without the namespace collision would be good.

> Patrick