Re: [myrinet] Re: [Myricom help #8219] Fwd: Registered memory in MPICHover GM

Bill Allard wrote:
> 
> Thanx, Patrick!  It seemed to me that MPICH-GM would <not> register memory
> each time a send or receive is issued and that some sort of caching
> mechanism would be necessary to avoid this.  This is what you affirmed and
> you even told me what source file to check it out in.

Right.

> In my own stuff, for the time being, I register a big block at startup,
> never register it again, and manage this block as needed.  

This is exactely what you should do, because you control everything in 
your application. The problem with MPI is that the application sends or 
receives message from buffers allocated statically or by malloc().

> It would be
> nice not to grab a big block in the beginning.  In order to know how to
> deal with all this, one needs to know just how expensive it is to register
> memory.  Is there a short answer to this?  If push comes to shove I can
> experiment but life is short...

Indeed :-)
The very short answer is: It depends of the CPU clock, SMP or not, 
Linux kernel release. 
The short answer is: roughly (5us (system call) + 0.5 us per page 
(lock+registration)).

Patrick

----------------------------------------------------------
|   Patrick Geoffray, Ph.D.      [email protected] 
|   Myricom, Inc.                http://www.myri.com
|   Cell:  865-389-8852          685 Emory Valley Rd (B)
|   Phone: 865-425-0978          Oak Ridge, TN 37830
----------------------------------------------------------