Re: [Myrinet] Performance degradation using gm_register_memory instead of gm_dma_calloc!

geoge_zhang wrote:

> using two processes and alloced DMA buffers using gm_register_memory and gm_dma_calloc
> seperately.The performance result is as follows.
> 1.use gm_register_memory register DMA buffer
> 2.allocate DMA buffer using gm_dma_calloc

>   As is shown in the above result, for small mesages, the performance is largely
> the same. But for larger messages, the performance of two DMA buffer request approaches
> is quite different.
>   We think the performance degradation of gm_register_memory may result from its more
> overhead of virtual-physical address transform.

What are you measuring ? Do you include the gm_register_memory() or 
the gm_dma_calloc() calls in the loop ? DMA-able memory allocation, 
either via gm_regiser_memory or gm_dma_alloc (BTW, why gm_dma_C_alloc ?) 
is expensive and should be avoided by allocating buffers at init time 
or implementing a registration cache. 

gm_dma_alloc uses the same code in the kernel than gm_register_memory 
(at least for Linux), so it should cost about the same. 
However, gm_dma_malloc behaves like a malloc and provide a little bit 
of caching and pre-allocation. This may be what you measure.

Patrick

----------------------------------------------------------
|   Patrick Geoffray, Ph.D.      [email protected] 
|   Myricom, Inc.                http://www.myri.com
|   Cell:  865-389-8852          685 Emory Valley Rd (B)
|   Phone: 865-425-0978          Oak Ridge, TN 37830
----------------------------------------------------------