RE: communication pattern causes corrupted data

David, 

Thanks for the info.  Is it only glibc that needs recompiling, or should
other system libraries be done as well?

-JE

-----Original Message-----
From: David B. Ritch
To: Pete Wyckoff
Cc: Joshua J. England; Patrick Geoffray; [email protected]; [email protected]
Sent: 3/23/2003 10:23 PM
Subject: Re: [Myrinet] communication pattern causes corrupted data

The glibc library, as distributed by RedHat, is compiled to run on *all*
alpha processors, including the original ev4, or 21064.  All the memory
operations available on the ev4 are word operations.  MPI-CH allocates
buffer space to messages on byte boundaries.  The ev4 has to write the
partial words of messages with a read-modify-write cycle.  This is not
atomic, and can lead to data corruption.  We were a little surprised to
see this on non-SMP machines, but the control flow can be interrupted in
the middle of a read-modify-write by a new message coming in.

The solution, on ev5 (21164) or newer alphas, is to recompile the
library specifically for an ev5 or newer.

dbr

On Sat, 2003-03-22 at 10:57, Pete Wyckoff wrote:
> [email protected] said on Sat, 22 Mar 2003 01:18 -0500:
> > I have tried your code on different configurations:
> > 
> > x86/Debian:
> > gm-1.6.3/gm-1.6.4-rc0, mpich-1.2.5..9/mpich-1.2.5..10-pre7
> > 
> > IA64/RH:
> > gm-1.6.3, mpich-1.2.5..9/mpich-1.2.5..10-pre7
> > 
> > It ran all night yesterday and all day today, and all of the runs
> > completed without errors. Is the probability of corruption large
enough
> > to happen at least once in one run ?
> 
> I too did 24 hours of non-stop runs on 16 processors (about half on 16
> dedicated nodes of 1 proc each, and half on 8 nodes using both procs)
> on:
> 
>     x86:  gm-1.5.1 mpich-1.2.4..8a
>     ia64: gm-1.6.3 mpich-1.2.4..8a
> 
> Compiled with gcc -g.  Both never caused errors using input.final.0.
> 
> Didn't the CPlant/NM guys have a similar issue with corruption on
alpha
> that ended up being a glibc issue?  I could be misremembering, but you
> might ask Ron.
> 
> 		-- Pete
> _______________________________________________
> Myrinet mailing list
> [email protected]
> http://email.osc.edu/mailman/listinfo/myrinet
-- 
David B. Ritch
High Performance Technologies, Inc.