Re: Rgerading RDMA/SDP and traditional socket apps

Caitlin,

I tend to agree with Mohan's observation. Preposted receive buffers allow
to eliminate a need in buffer advertising, but it still involves delay. I
can see two options: option a) RNIC does not keep any intermediate buffer,
and then it needs to wait to Ack to arrive, before releasing consumer
buffer, option b) RNIC keeps intermediate buffers, and then consumer
buffer is released after it is copied to the intermediate buffer. In both
cases we have delay involved in blocking send operation, plus some sort of
hw/sw sync/coordination to report completion of operation and release a
buffer.

Vadim




|---------+---------------------------->
|         |           Caitlin Bestler  |
|         |           <[email protected]> |
|         |           Sent by:         |
|         |           rddp-bounces@ietf|
|         |           .org             |
|         |                            |
|         |                            |
|         |           04/10/04 06:46   |
|---------+---------------------------->
  >-------------------------------------------------------------------------------------------------------------------------------|
  |                                                                                                                               |
  |       To:       RDDP <[email protected]>                                                                                          |
  |       cc:       Mohan Parthasarathy <[email protected]>                                                                     |
  |       Subject:  Re: [rddp] Rgerading RDMA/SDP and traditional socket apps                                                     |
  >-------------------------------------------------------------------------------------------------------------------------------|





On Oct 4, 2004, at 5:04 AM, Mohan Parthasarathy wrote:

>  Hi Mike/Caitlin,
>
>    Thanks for the answers. So, what I understand is that my assumption
>    that existing socket apps might not have ownership of the buffer
> when
>    the send completes, was wrong.
>    But, doesn't this mean that send calls will take more time to
> complete,
>    if the RNIC has to DMA the buffers before control to the buffer is
>    returned to the caller. Are there any potential performance issues
> here
>    for existing synchronous sockets ?
>
>  Regards,
>  Mohan


SDP defines multiple transfer procedures precisely to avoid
this problem.  SDP should be configured so that zero-copy
transfers are only done when doing so will improve performance.

For example,  when the receive  is posted long enough before
the send for the data sink buffer to have been advertised the
data source can RDMA Write to the final target buffer without
any delay.

SDP inherently involves more wire traffic, but depending on
the timing of sends and receives this extra traffic may have no
impact on latency.  The benefits of direct placement should
overcome any penalty for the wire traffic.  But that penalty is
there, which is why local-only optimizations that achieve direct
placement without wire traffic might be more efficient, and why
full adaptation to RDMA-aware application logic will almost
always be more efficient.



_______________________________________________
rddp mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rddp