Re: [IPFIX] FW: UDP Sequence Numbers rfc 5101

Paul 
Two things:
1/ Thanks for the explanation.  The assessment of relative importance of
template loss v count of data records is clearly case depended, but
reasonable.
2/ Sentence three remains incorrect  to my mind.  It is a Recommendation
that is impossible to fulfil .  
As it stands the sentence leads the reader to believe that IPFIX over UDP
provides greater reliability than it actually does.

Regards
Steve

-----Original Message-----
From: Paul Aitken [mailto:[email protected]] 
Sent: 03 April 2013 11:42
To: [email protected]
Cc: [email protected]
Subject: Re: [IPFIX] FW: UDP Sequence Numbers rfc 5101

Steve,

The sequence number change was a deliberate decision in RFC 5101, so that
collectors can use the sequence numbers to determine (and thus
report) the number of lost data records.

This is an improvement of RFC 3954, where the sequence number only counted
export packets. In this case it's impossible to determine how many data
records were lost, ie the amount of loss is unquantifiable. 
The packets could have contained templates and/or data records; we simply
don't know.

Neither of these sequence numbers helps to specifically determine whether a
template has been lost. 3954 sequence numbers identify packet loss without
indicating whether it's template or data record. 5101 sequence numbers
identify data record loss only.

Identifying template loss would require an additional template sequence
number.

If loss is a big issue in your network, don't export over UDP. If you must
export over UDP, and loss is an issue, then don't redefine templates on the
fly.

In my experience templates are generally static (defined once and never
withdrawn or redefined), and limited to a few 10's in number. However, there
are 65280 templates available. Defining these incrementally at 1000/sec with
a 1-minute lifetime, or at 100/sec with a 10-minute lifetime, ensures that
no template will ever be redefined within its lifetime - although I can't
think of any use case that requires (re)defining templates that quickly.

P.


On 03/04/13 10:52, Steve Nash wrote:
> The third sentence remains incorrect.  Only IPFIX Data Records are 
> protected by sequence number .  "IPFIX messages" in general are not 
> protected (even templates).
> This is a retrograde step from rfc3954.
> It makes use of IPFIX for flow reporting over UDP less tolerant than 
> rfc 3954.  The unrecognised loss of a template change message could 
> result in significant misinterpretation of data, which is worse than 
> just recognised data loss.
>
> The second sentence seems redundant, as UDP is no longer different to 
> TCP and SCTP.  You can avoid any mention of transport protocol here.
>
> Regards
> Steve
>
>
> -----Original Message-----
> From: Brian Trammell [mailto:[email protected]]
> Sent: 25 March 2013 00:32
> To: [email protected]
> Cc: [email protected]
> Subject: Re: [IPFIX] UDP Sequence Numbers rfc 5101
>
> Hi, Steve,
>
> This is an error in RFC 5101; the intent of the Sequence Number is 
> that it be Observation Domain (and in the case of SCTP, Stream) scoped 
> (i.e., you are correct that transport independence, to the extent 
> possible, is the intent). The language in the present revision of 
> 5101bis still contains the same error. Thanks for the catch!
>
> I would suggest we fix this by replacing the paragraph in question as
> follows:
>
> OLD:
>
>     The Collecting Process SHOULD deduce the loss and reordering of IPFIX
>     Data Records by looking at the discontinuities in the IPFIX Sequence
>     Number.  In the case of UDP, the IPFIX Sequence Number contains the
>     total number of IPFIX Data Records sent for the UDP Transport Session
>     prior to the receipt of this IPFIX Message, modulo 2^32.  A Collector
>     SHOULD detect out-of-sequence, dropped, or duplicate IPFIX Messages
>     by tracking the Sequence Number.
>
> NEW:
>
>     The Collecting Process SHOULD deduce the loss and reordering of IPFIX
>     Data Records by looking at the discontinuities in the IPFIX Sequence
>     Number.  In the case of UDP, the IPFIX Sequence Number contains the
>     total number of IPFIX Data Records sent for the Observation Domain in
the
>     current UDP Transport Session prior to the receipt of this IPFIX 
> Message,
>
>     modulo 2^32.  A Collector SHOULD detect out-of-sequence, dropped, or
>     duplicate IPFIX Messages by tracking the Sequence Number.
>
> The document is presently in Publication Requested state, so I think 
> this should be placed as an RFC editor note on the document. WG: any 
> comments thereon?
>
> Best regards,
>
> Brian
>
> On 22 Mar 2013, at 0:45, Steve Nash <[email protected]> wrote:
>
>> Please consider the wording of the paragraph on Sequence Numbers in
> Section 3.1.
>>        Incremental sequence counter modulo 2^32 of all IPFIX Data Records
>>        sent in the current stream from the current Observation Domain by
>>        the Exporting Process. Each SCTP Stream counts sequence numbers
>>        separately, while all messages in a TCP connection or UDP
>>        transport session are considered to be part of the same stream.
>> The phrase I have put in italics is depended on Transport protocol.
>> Section 10.3.2 Reliability says:
>>   
>> In the case of UDP, the IPFIX Sequence Number contains the total 
>> number of IPFIX Data Records sent for the UDP Transport Session prior 
>> to the receipt of this IPFIX Message
>>   
>> This is NOT dependent on the ObservationDomain.  Nothing prohibits 
>> the
> Exporting Process from using the same Transport Session for multiple 
> Observation Domains.  But in UDP the Sequence number is for the 
> Transport session, whereas in TCP and SCTP each Observation Domain has 
> an independent sequence on the same  (and on each) Transport Session.
>>   
>> The term 'Stream' seems to take different meanings in IPFIX, 
>> depending on
> the transport protocol.
>> This is not consistent with IPFIX being  "Transport protocol independent
"
> (Section 10 first para).
>>   
>> Regards
>> Steve Nash
>> [email protected]
>> UK
>> _______________________________________________
>> IPFIX mailing list
>> [email protected]
>> https://www.ietf.org/mailman/listinfo/ipfix
> _______________________________________________
> IPFIX mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix