Re: gm_dma_malloc

Hi,

Gyu Sang Choi wrote:
> As I know, "gm_dma_malloc" mmaps  user-space memory into NIC memory.
> Thus, I write something on mmaped memory, and it automatically updates 
> NIC memory.

No, gm_dma_malloc does not do that. It allocates host memory and 
register it (pin it for DMA access). It is semantically equivalent to a 
simple malloc() followed by a gm_register_memory(). DMA-able memory 
means that the NIC can access it by DMA, that's all.

> My myrinet card has 8MBytes on-chip memory.
> As I understand, I can only mmap  8MBytes user-space memory because my 
> NIC has only 8 Mbytes.

No, the NIC memory is used for the firmware code and data, and some 
buffering, not to map the user space memory.

> According to your FAQ, I can mmap 7/8 physical memory.
> How can I mmap 7/8 physical system memory, while my NIC has only 8 Mbytes?

You can *register* 7/8 of the physical memory, not *mmap* it. When you 
register memory, the driver pins the pages and add their physical 
addresses in a hash table in the driver. Nothing goes to the NIC then. 
When the NIC needs to access memory that has been registered, it fetches 
the physical address from this hash (well, actually it is more 
complicated because there is a cache in the NIC).

> Is there anyway to access directly 8MBytes on-chip memory from a user 
> application?

Not without help from the driver. It's not hard to do though. Actually, 
the driver already maps a little bit of NIC memory into user-space for 
implementing the send queue for example.

What do you want to do with access to the memory on the NIC ?

Patrick
-- 

Patrick Geoffray
Myricom, Inc.
http://www.myri.com