Re: [myrinet] Re: Sender masks for gm_provide_recv_buffer*

Hi Bob,

Yep, this would work - I'll give it a go. 

Are any potential performance gotcha's in how the dma transfers
are set up? I've a slight worry that the 2^(size) bytes
might be dma'd to the LANai and then the relevant portion sent, but 
I know precious little about PCI dma constraints so I'm probably
just being paranoid.

Cheers, 

Peter

On Wed, 9 Aug 2000, Bob Felderman wrote:

> => For latency hiding reasons (among others), it is beneficial initiate 4
> => communications at a time, one for each dimension. Forcing a desired 
> => receive buffer is problematic, and forced me to use the directed send.
> => However, as I read the documentation completion notification is
> => unavailable, and I am following the send up with a small "ping" message to
> => generate recv events.
> => 
> => I'd like to get rid of this extra send, while still maintaining
> => the current zero copy status of the code, and there are two obvious
> => solutions - 
> 
> 
> How about taking advantage of an undocumented feature of 
> the "size" field in the gm_send / gm_receive and eliminate
> the use of gm_directed_send?
> 
> If your messages are not too large, say less than 1meg, then
> you can use the size parameter as a sort of tag to 
> match your receives and sends.
> The MCP does not verify that the length of a large
> buffer is less than 2^^size. 
> 
> just define 4 sizes for N,E,W,S
> #define NORTH 21
> #define EAST  22
> #define WEST  23
> #define SOUTH 24
> 
> Then, for each iteration, use a 
> gm_provide_receive_buffer(port, &data__from_north, NORTH, priority);
> gm_provide_receive_buffer(port, &data_from_south, SOUTH, priority);
> gm_provide_receive_buffer(port, &data_from_east , EAST, priority);
> gm_provide_receive_buffer(port, &data_from_west , WEST, priority);
> 
> and a 
> gm_send_with_callback(port, &data_to_north,  NORTH, len,priority,node,port_id,...)
> gm_send_with_callback(port, &data_to_south,  SOUTH, len,priority,node,port_id,...)
> gm_send_with_callback(port, &data_to_east,  EAST, len,priority,node,port_id,...)
> gm_send_with_callback(port, &data_to_west,  WEST, len,priority,node,port_id,...)
> 
> 
> You can obviously provide more receive buffers of each size 
> if needed to improve the buffering/bandwidth.
> 
> Will this work for your application?
> 
> Bob
> 
>