Re: [Myrinet] [Q] VIGM performance

Hi Moon-Sang,

On Tue, 2002-09-10 at 03:59, Moon-Sang Lee wrote:

- Blocking case:
> Size (Bytes)     Latency (us)    Bandwidth (MB/s) (bps)
> 4               62.60           0.06
- Polling case:
> Size (Bytes)     Latency (us)    Bandwidth (MB/s) (bps)
> 4               14.45           0.26

> I got a similar result to yours in case of a non-blocking test. 
> But I think it is not fair to use a non-blocking test for measuring 
> the latency. For a correct pingpong test, there should be only 
> a single packet in the Myrinet link. Using a non-blocking test issues
> as many packets as possible to the Myrinet link all at once. 
> (it better fits to an effective bandwidth test, I think). 

No, there is always a single packet on the fly at one time. The
difference between non-blocking and blocking test is not at all on
flooding the link or not. 
The blocking test uses VipSendWait/VipRecvWait to wait for the
completion of the sends and the receives. In this case, the NIC is asked
to raise an interrupt when the work is finished, and the process is
unscheduled and goes to sleep in the kernel, at each iterations !
So you pay interrupt + context switches in the 62 us.
The non-blocking test is also pushing one packet at a time on the link,
but it checks for send or receive completion by polling (while loops
with VipSendDone and VipRecvDone).

Some people want to look at the blocking case because they use many
threads per processor, they expect the threads to be scheduled when
waiting for operation completion and they want to know how large is the
overhead. If you are using one process/thread per processor, the polling
case is what you are looking for.

Finally, the difference between your 14 us and my 10 is certainly due to
Lanai 4 or 7 and Lanai9 clocks.

Patrick
-- 
----------------------------------------------------------
|   Patrick Geoffray, Ph.D.      [email protected] 
|   Myricom, Inc.                http://www.myri.com
|   Cell:  865-389-8852          685 Emory Valley Rd (B)
|   Phone: 626-821-5555          Oak Ridge, TN 37830
----------------------------------------------------------