Re: [myrinet] Re: myrinet and one-sided messaging

Hi Mario,

We were considering FM as an alternative protocol to GM 
in our work on ARMCI for its certain advantages over GM
e.g., excellent support for noncontiguous data transfers and better
for one-sided messaging. However, a majority of Myrinet sites 
we knew in fact are interested in running production rather 
than research network drivers & communication s/w. Given the fact 
that FM is not compatible with the standard GM installations and 
at this time it does not support operating systems other than 
Windows we really had no choice but to target GM.

I also have a comment/question about the latency numbers that you
mentioned for get. If I remember from some previous paper on HPVM
(HPDC'98?), the low latency in SHMEM/GM was achieved by dedicating 
a CPU on an SMP node to service one-sided communication requests. 
For example, on a dual CPU SMP node, only one CPU would run the 
application code and the other an extra FM thread. This thread was 
actively polling the network interface thus allowing FM to avoid 
the cost of interrupt processing that in the current ARMCI/GM 
implementation adds tens of microseconds to the get latency. 
Can you confirm that the latency number you quoted in fact 
corresponds to the configuration with a dedicated CPU actively 
polling? Also what was the get b/w -- you mentioned put b/w and 
get latency?

Regards,

Jarek



Mario Lauria wrote:
> 
> Bob is right - SHMEM is one of the libraries ported on top
> of Fast Messages (FM) as part of the HPVM project. Highligths
> of the SHMEM-FM implementation are 23 usec min latency for a get
> operation, and 93MB/s peak bandwidth for put. The pointer to
> Andrew Chien's group site is http://www-csag.ucsd.edu
> 
> The current version of HPVM runs under Windows NT and requires
> the old Myrinet boards (M2M). I am a former member of the CSAG group
> and if I find enough interest in an M3M and/or Linux version of HPVM I
> could ask Andrew to let me update and redistribute the code.
> 
>         Mario.
> 
> > Date: Wed, 10 Jan 2001 11:11:14 -0800 (PST)
> > From: Bob Felderman <[email protected]>
> > Subject: Re: [myrinet] myrinet and one-sided messaging
> 
> > gm-2.0 will support
> > gm_put (formerly gm_directed_send)
> > gm_get (not implemented before gm-2.0)
> 
> > with those two functions, it should be simple to get shmem
> > ported to GM.
> 
> > I think HPVM from Andrew Chien's group at UCSD does
> > have a shared memory option.
> 
> --
> -=-=-=-=-=- Mario Lauria, Assistant Professor -=-=-=-=-=-
> Department of Computer     Email: [email protected]
>  and Information Science   Phone: (614) 292-7027
> The Ohio State University  Fax:   (614) 292-2911
> Columbus, OH 43210-1277    Radio: N9ZCT
> -=-=-=-=- http://www.cis.ohio-state.edu/~lauria -=-=-=-=-