[Myrinet] Performance degradation using gm_register_memory instead of gm_dma_calloc!

Hi,everyone:
   Our group is now testing the performance of GM-1.5 on our Linux PC cluster
connected with Myrinet 2000. We tested the point-to-point bandwidth performance 
using two processes and alloced DMA buffers using gm_register_memory and gm_dma_calloc
seperately.The performance result is as follows.
1.use gm_register_memory register DMA buffer
Loop time is: 10000
Average loopback bandwidth is: (byte) (MB/s)
1  0.1
2  0.1
4  0.3
8  0.6
16  1.2
32  2.4
64  4.7
128  8.7
256 15.4
512 24.9
1024 42.4
2048 66.6
4096 88.6
8192 126.0
16384 154.8
32768 174.9

2.allocate DMA buffer using gm_dma_calloc
Loop time is: 10000
Average loopback bandwidth is: (byte) (MB/s)
1   0.1
2   0.2
4   0.3
8   0.6
16   1.2
32   2.4
64   4.7
128   8.7
256  15.9
512  25.6
1024  43.2
2048  67.3
4096  91.9
8192  134.1
16384  173.7
32768  202.7

  As is shown in the above result, for small mesages, the performance is largely 
the same. But for larger messages, the performance of two DMA buffer request approaches
is quite different.
  We think the performance degradation of gm_register_memory may result from its more 
overhead of virtual-physical address transform.
  Does anyone can explain something more about this topic? Any information will be greatly
appreciated!

Best regards
Geoge

Dept. of Computer Science and Technology.
Tsinghua University.