Re: [Myrinet] bogus page

Salut Thomas

Thomas Guignon wrote:
> 
> Hello,
> Sometime our MPI application makes the gm driver write something like this:
> GM: LANAI[0]: port=2 bogus page SEND ptr = 0x40667020 FAILED  (maxidx=32767)

It happens when the Lanai is trying to resolve a virtual address and this 
address is not in the registration hash, i.e. is not registered. There is 2 
causes for that, a good one and a bad one. The bad one signals a bug in the 
application, the send buffer covers a page (at least one) that is not registered 
or allocated with gm_dma_malloc. It can happen on the receive side also. The 
ptr is the virtual page address. The second case is when the GM port is closed 
when messages are in flight: the port closure will deregistred automatically 
all registered memory, but it may happen in the middle of a large message send 
processed on the Lanai at that time. The fist pages are Ok, the rest of the 
buffer appears deregistered and you have this message. That would happen when 
the process dies, segfaults, killed by a signal, etc.
It's a good timing as I have just finished a change for GM-1.5.1 that would 
remove the good case: if the port is closed, no messages are printed in the 
log. So in GM-1.5.1, this type of message would indicates a bug, 100% sure.

Now, if you see this message when the application is running and the port is 
open (and stays open), I would like to run your MPI code to see what's
happening.

> I think this was caused by the fact that the
> buffer was in swap instead of physical memory, correct me if I am wrong.

Registered memory is locked in physical memory, and to register pages force 
them to swap out if they are swapped, so the buffer cannot be in swap.

> Does --gm-can-register-memory option forces buffer to be in DMA-able
> memory?

Are you using mpich-gm 1.2..X ? there is no more "--gm-can-register-memory" 
in 1.2.1..X

Cordialement.

Patrick

----------------------------------------------------------
|   Patrick Geoffray, Ph.D.      [email protected] 
|   Myricom, Inc.                http://www.myri.com
|   Cell:  865-389-8852          685 Emory Valley Rd (B)
|   Phone: 865-425-0978          Oak Ridge, TN 37830
----------------------------------------------------------