Clause 3 comments for draft-ietf-rddp-ddp-01

Below are comments on clause 3 of draft-ietf-rddp-ddp-01

There are 10 Editorial comments here all dealing with editorial nits.
This is only of interest to the editor(s).


Comments are given in the following form:

Number, type, significance, reference
Comment

CLAUSE 3

1, Editorial, Low 3 1st PP, page 3

Is: Direct Data Placement Protocol (DDP) enables an Upper Layer Protocol

   (ULP) to send data to a Data Sink without requiring the Data Sink to 
   Place the data in an intermediate buffer - thus when the data 
   arrives at the Data Sink, the network interface can Place the data 
   directly into the ULP's buffer. This can enable the Data Sink to 
   consume substantially less memory bandwidth than a buffered model 
   because the Data Sink is not required to move the data from the 
   intermediate buffer to the final destination. Additionally, this can 
   also enable the network protocol to consume substantially fewer CPU 
   cycles than if the CPU was used to move the data, and removes the 
   bandwidth limitation of only being able to move data as fast as the 
   CPU can copy the data. 

Suggest: Direct Data Placement Protocol (DDP) enables an Upper Layer
Protocol 
   (ULP) to send data to a data sink without requiring the data sink to 
   place the data in an intermediate buffer - thus when the data 
   arrives at the data sink, the network interface can place the data 
   directly into the ULP's buffer. This can enable the data sink to 
   consume substantially less memory bandwidth than a buffered model 
   because the data sink is not required to move the data from the 
   intermediate buffer to the final destination. Additionally, this can 
   also enable the network protocol to consume substantially fewer CPU 
   cycles than if the CPU was used to move the data, and removes the 
   bandwidth limitation of only being able to move data as fast as the 
   CPU can copy the data. 
Changes: Removed capitalization on "data sink" and "place"

2, Editorial, Low 3.1 1st bullet page 4

Is:     
* Provide a buffer model that enables the Local Peer to Advertise 
        a named buffer (i.e. a Tag for a buffer) to the Remote Peer, 
        such that across the network the Remote Peer can Place data 
        into the buffer at Remote Peer specified locations. This is 
        referred to as the Tagged Buffer Model. 

Suggest:     
* Provide a buffer model that enables the local peer to advertise 
        a named buffer (i.e. a tag for a buffer) to the remote peer, 
        such that across the network the remote peer can place data 
        into the buffer at remote peer specified locations. This is 
        referred to as the tagged buffer model. 
Changes: removed capitalization of "local peer", "advertise", "remote
peer", tagged buffer model, and "place".

3, Editorial, Low 3.1, 2nd bullet page 4

Is: 
     * Provide a second receive buffer model which preserves ULP 
        message boundaries from the Remote Peer and keeps the Local 
        Peer's buffers anonymous (i.e. Untagged). This is referred to 
        as the Untagged Buffer Model. 
Suggest:
     * Provide a second receive buffer model which preserves ULP 
        message boundaries from the remote peer and keeps the local 
        peer's buffers anonymous (i.e. untagged). This is referred to 
        as the untagged buffer model.
Changes: removed capitalization

4, Editorial, Low 3.1, 3rd bullet page 4

Is:
     * Provide reliable, in-order Delivery semantics for both Tagged 
        and Untagged Buffer Models.  
Suggest:
     * Provide reliable, in-order delivery semantics for both Tagged 
        and untagged buffer models.
Changes: Capitalization

5, Editorial, Low 3.1, 5th bullet page 4

Is:
     * Enable the ULP buffer to be used as a reassembly buffer, 
        without a need for a copy, even if incoming DDP Segments arrive 
        out of order. This requires the protocol to separate Data 
        Placement of ULP Payload contained in an incoming DDP Segment 
        from Data Delivery of completed ULP Messages. 
Suggest:
     * Enable the ULP buffer to be used as a reassembly buffer, 
        without a need for a copy, even if incoming DDP segments arrive 
        out of order. This requires the protocol to separate data 
        placement of ULP payload contained in an incoming DDP segment 
        from data delivery of completed ULP messages.
Change: Capitalization

6 Editorial, Low 3.1 6th bullet page 4

Is:
     * If the LLP supports multiple LLP streams within a LLP 
        Connection, provide the above capabilities independently on 
        each LLP stream and enable the capability to be exported on a 
        per LLP stream basis to the ULP. 

Suggest:
     * If the LLP supports multiple LLP streams within a LLP 
        connection, provide the above capabilities independently on 
        each LLP stream and enable the capability to be exported on a 
        per LLP stream basis to the ULP.
Change: Capitalization

7 Editorial, Low 3.2 1st three pps page 4

Is:
DDP supports two basic data transfer models - a Tagged Buffer data 
   transfer model and an Untagged Buffer data transfer model.  

   The Tagged Buffer data transfer model requires the Data Sink to send 
   the Data Source an identifier for the ULP buffer, referred to as a 
   Steering Tag (STag). The STag is transferred to the Data Source 
   using a ULP defined method. Once the Data Source ULP has an STag for 
   a destination ULP buffer, it can request that DDP send the ULP data 
   to the destination ULP buffer by specifying the STag to DDP. Note 
   that the Tagged Buffer does not have to be filled starting at the 
   beginning of the ULP buffer. The ULP Data Source can provide an 
   arbitrary offset into the ULP buffer.  

   The Untagged Buffer data transfer model enables data transfer to 
   occur without requiring the Data Sink to Advertise a ULP Buffer to 
   the Data Source. The Data Sink can queue up a series of receive ULP 
   buffers. An Untagged DDP Message from the Data Source consumes an 
   Untagged Buffer at the Data Sink. Because DDP is message oriented, 
   even if the Data Source sends a DDP Message payload smaller than the 
   receive ULP buffer, the partially filled receive ULP buffer is 
   Delivered to the ULP anyway. If the Data Source sends a DDP Message 
   payload larger than the receive ULP buffer, it results in an error.  

   There are several key differences between the Tagged and Untagged 
   Buffer Model: 
 
Suggest:
DDP supports two basic data transfer models - a tagged buffer data 
   transfer model and an untagged buffer data transfer model.  

   The tagged buffer data transfer model requires the data sink to send 
   the data source an identifier for the ULP buffer, referred to as a 
   Steering Tag (STag). The STag is transferred to the data source 
   using a ULP defined method. Once the data source ULP has an STag for 
   a destination ULP buffer, it can request that DDP send the ULP data 
   to the destination ULP buffer by specifying the STag to DDP. Note 
   that the tagged buffer does not have to be filled starting at the 
   beginning of the ULP buffer. The ULP data source can provide an 
   arbitrary offset into the ULP buffer.  

   The untagged buffer data transfer model enables data transfer to 
   occur without requiring the data sink to advertise a ULP buffer to 
   the data source. The data sink can queue up a series of receive ULP 
   buffers. An untagged DDP message from the data source consumes an 
   untagged buffer at the data sink. Because DDP is message oriented, 
   even if the data source sends a DDP message payload smaller than the 
   receive ULP buffer, the partially filled receive ULP buffer is 
   delivered to the ULP anyway. If the data source sends a DDP message 
   payload larger than the receive ULP buffer, it results in an error.  

   There are several key differences between the tagged and untagged 
   buffer model: 

Changes: Capitalization

8 Editorial, Low, 3.2, bullets on page 5

Is:
     * For the Tagged Buffer Model, the Data Source specifies which 
        received Tagged Buffer will be used for a specific Tagged DDP 
        Message (sender-based ULP buffer management). For the Untagged 
        Buffer Model, the Data Sink specifies the order in which 
        Untagged Buffers will be consumed as Untagged DDP Messages are 
        received (receiver-based ULP buffer management). 

     * For the Tagged Buffer Model, the ULP at the Data Sink must 
        Advertise the ULP buffer to the Data Source through a ULP 
        specific mechanism before data transfer can occur. For the 
        Untagged Buffer Model, data transfer can occur without an end-
        to-end explicit ULP buffer Advertisement. Note, however, that 
        the ULP needs to address flow control issues. 

     * For the Tagged Buffer Model, a DDP Message can start at an 
        arbitrary offset within the Tagged Buffer. For the Untagged 
        Buffer Model, a DDP Message can only start at offset 0. 

     * The Tagged Buffer Model allows multiple DDP Messages targeted 
        to a Tagged Buffer with a single ULP buffer Advertisement. The 
        Untagged Buffer Model requires associating a receive ULP buffer 
        for each DDP Message targeted to an Untagged Buffer.  
Suggest:
     * For the tagged buffer model, the data source specifies which 
        received tagged buffer will be used for a specific tagged DDP 
        message (sender-based ULP buffer management). For the untagged 
        buffer model, the data sink specifies the order in which 
        untagged buffers will be consumed as untagged DDP messages are 
        received (receiver-based ULP buffer management). 

     * For the tagged buffer model, the ULP at the data sink must 
        advertise the ULP buffer to the data source through a ULP 
        specific mechanism before data transfer can occur. For the 
        untagged buffer model, data transfer can occur without an end-
        to-end explicit ULP buffer advertisement. Note, however, that 
        the ULP needs to address flow control issues. 

     * For the tagged buffer model, a DDP message can start at an 
        arbitrary offset within the tagged buffer. For the untagged 
        buffer model, a DDP message can only start at offset 0. 

     * The tagged buffer model allows multiple DDP messages targeted 
        to a tagged buffer with a single ULP buffer advertisement. The 
        untagged buffer model requires associating a receive ULP buffer 
        for each DDP message targeted to an untagged buffer. 
Change: Capitalization

9 Editorial, Low 3.2, text following bullets to end of clause

Is:
Either data transfer model Places a ULP Message into a DDP Message. 
   Each DDP Message is then sliced into DDP Segments that are intended  
 
shah, et. al.             Expires April 2004                         5 
 
Internet-Draft        DDP Protocol Specification        October 2003 
 
   to fit within a lower-layer-protocol's (LLP) Maximum Upper Layer 
   Protocol Data Unit (MULPDU). Thus the ULP can post arbitrary size 
   ULP Messages, containing up to 2^32 - 1 octets of ULP Payload, and 
   DDP slices the ULP message into DDP Segments which are reassembled 
   transparently at the Data Sink.  

   DDP provides in-order Delivery for the ULP. However, DDP 
   differentiates between Data Delivery and Data Placement. DDP 
   provides enough information in each DDP Segment to allow the ULP 
   Payload in each inbound DDP Segment payloads to be directly Placed 
   into the correct ULP Buffer, even when the DDP Segments arrive out-
   of-order. Thus, DDP enables the reassembly of ULP Payload contained 
   in DDP Segments of a DDP Message into a ULP Message to occur within 
   the ULP Buffer, therefore eliminating the traditional copy out of 
   the reassembly buffer into the ULP Buffer. 

   A DDP Message's payload is Delivered to the ULP when: 

     * all DDP Segments of a DDP Message have been completely received 
        and the payload of the DDP Message has been Placed into the 
        associated ULP Buffer, 

     * all prior DDP Messages have been Placed, and 

     * all prior DDP Message Deliveries have been performed. 

   The LLP under DDP may support a single LLP stream of data per 
   connection (e.g. TCP) or multiple LLP streams of data per connection 
   (e.g. SCTP). But in either case, DDP is specified such that each DDP 
   Stream is independent and maps to a single LLP stream. Within a 
   specific DDP Stream, the LLP Stream is required to provide in-order, 
   reliable Delivery. Note that DDP has no ordering guarantees between 
   DDP Streams. 

   A DDP protocol could potentially run over reliable Delivery LLPs or 
   unreliable Delivery LLPs. This specification requires reliable, in 
   order Delivery LLPs. 
Suggest:
Either data transfer model places a ULP message into a DDP message. 
   Each DDP message is then sliced into DDP segments that are intended  
 
shah, et. al.             Expires April 2004                         5 
 
Internet-Draft        DDP Protocol Specification        October 2003 
 
   to fit within a lower-layer-protocol's (LLP) Maximum Upper Layer 
   Protocol Data Unit (MULPDU). Thus the ULP can post arbitrary size 
   ULP messages, containing up to 2^32 - 1 octets of ULP payload, and 
   DDP slices the ULP message into DDP segments which are reassembled 
   transparently at the data sink.  

   DDP provides in-order delivery for the ULP. However, DDP 
   differentiates between data delivery and data placement. DDP 
   provides enough information in each DDP Segment to allow the ULP 
   payload in each inbound DDP segment payloads to be directly placed 
   into the correct ULP Buffer, even when the DDP Segments arrive out-
   of-order. Thus, DDP enables the reassembly of ULP payload contained 
   in DDP segments of a DDP message into a ULP message to occur within 
   the ULP buffer, therefore eliminating the traditional copy out of 
   the reassembly buffer into the ULP buffer. 

   A DDP message's payload is delivered to the ULP when: 

     * all DDP segments of a DDP message have been completely received 
        and the payload of the DDP message has been placed into the 
        associated ULP buffer, 

     * all prior DDP messages have been placed, and 

     * all prior DDP message deliveries have been performed. 

   The LLP under DDP may support a single LLP stream of data per 
   connection (e.g. TCP) or multiple LLP streams of data per connection 
   (e.g. SCTP). But in either case, DDP is specified such that each DDP 
   stream is independent and maps to a single LLP stream. Within a 
   specific DDP stream, the LLP stream is required to provide in-order, 
   reliable delivery. Note that DDP has no ordering guarantees between 
   DDP streams. 

   A DDP protocol could potentially run over reliable delivery LLPs or 
   unreliable delivery LLPs. This specification requires reliable, in 
   order delivery LLPs.
Changes: Capitalization

10 Editorial Low 3.3 text before figure 1

Is:
   DDP enables direct data Placement capability for any ULP, but it has 
   been specifically designed to work well with RDMAP (see [RDMA]), and 
   is part of the iWARP protocol suite.  
Suggest:
   DDP enables direct data placement capability for any ULP, but it has 
   been specifically designed to work well with RDMAP (see [RDMA]), and 
   is part of the iWARP protocol suite. 
Change: Capitalization of "placement" 
 

Barry Reinhold
Lamprey Networks
[email protected]
(603) 868-8411