Re: [IPFIX] draft-ietf-ipfix-protocol-rfc5101bis-03

Hi Brian, all,

That all makes sense.  I particularly like the addition of a disclaimer.

-Andrew


On 12/05/2012 05:59 AM, Brian Trammell wrote:
> hi, Paul, Andrew, all,
>
> Agreed, though it is not clear to me how to really account for time at receiver here. Actually, I think this makes more sense to explain in terms of "template lifetime" as opposed to "time of effect of a template or withdrawal"... How about the following?
>
> NEW:
>
>    Put another way, a Template describes Data Records contained in IPFIX Messages
>    with an Export Time between the Export Time of the IPFIX Message containing the
>    Template Record and the end of the Transport Session, inclusive, if that
>    Template is not subsequently withdrawn.
>
>    If a Template is subsequently withdrawn, then that Template describes Data
>    Records contained in IPFIX Messages with an Export Time between the Export Time
>    of the IPFIX Message containing the Template Record and the Export Time of the
>    IPFIX Message containing the Template Withdrawal, inclusive.
>
> Stepping back further, the real issue here is that you can only have two of the following three things:
>
> (1) template identifier reuse,
>
> (2) template transmission without reliable ordering (i.e., templates in multiple SCTP streams, or UDP as a transport protocol),
>
> (3) verifiably correct interpretation of data records.
>
> I strongly suspect this is why we have seen no interest in five years of interoperability testing in (1), and interest in (2) only because IPFIX is envisioned for deployment to replace UDP transport of older versions of NetFlow, where the network is either provisioned to minimize loss, or the processes around the collection and processing of the data have been adjusted to the fact that UDP is simply not reliable.
>
> And it looks we fell into the trap again, just like we did in 2007 with 5101, of trying to pretend this is a fixable problem. So I think in addition we probably need a warning label on template reuse -- even if you follow all these rules, a stream of IPFIX messages compliant with the protocol may come out the other side of the network in an order which makes it difficult to determine what the exporter meant, so practically speaking reuse needs various knobs that admins can tune to determine timing delays for reuse, and those admins need to be conservative when tweaking them, or tolerant of lost or corrupted data. I'd replace the final paragraph of section 8.2 with an actual warning; something like:
>
> NEW:
>
>      Note that, even if sent in-order, IPFIX Messages containing Template
>      management actions could arrive at the Collecting Process out-of-order,
>      i.e. if sent via UDP or via different SCTP streams. Template Withdrawals
>      and subsequent reuse of Template IDs, in particular, can significantly
>      complicate the problem of determining Template lifetimes at the Collecting
>      Process. Therefore, Collecting Processes MAY implement a buffer to handle
>      out-of-order Template management events; this buffer, if implemented,
>      SHOULD be configurable to impart a delay on the order of the maximum
>      reordering delay experienced at the Collecting Process.
>
> This, I think, handles the reception order problem, as well.
>
> Thoughts?
>
> Best regards,
>
> Brian
>
> On 4 Dec 2012, at 21:11, Paul Aitken <[email protected]> wrote:
>
>> Andrew, all,
>>
>> There's a loophole in both definitions: if a collector receives data records with export time = T2, then a Template with export time = T1 (where T1 <= T2), the Template applies to the previously-received data records. This could be due to UDP out of order, or delivery in different SCTP streams.
>>
>> However when the collector receives data records at T1 it doesn't know that a template is coming at T2, so its behaviour is unclear, especially if it already has a template for those data records. That could be an earlier template which no longer applies.
>>
>> So the definition might benefit from some words about reception order.
>>
>> P.
>>
>>
>>
>> On 04/12/12 18:22, Andrew Feren wrote:
>>> Hi All, Atsushi,
>>>
>>> On 11/25/2012 08:41 AM, ATSUSHI KOBAYASHI wrote:
>>>> Dear all, Nevil,
>>>>
>>>> I checked the draft, I found some comments and editorial issue.
>>>> Please see in-line.
>>>>
>>>> Regards, Atsushi
>>> [ snip ]
>>>> Comment#3:
>>>>
>>>> I could not understand the following paragraph. If you want to describe
>>>> another way, please explain it in more detail.
>>> I think Atsushi makes a good point here.
>>>>>     Put another way, a Template only describes Records contained in IPFIX
>>>>>     Messages with the same Export Time as the IPFIX Message containing
>>>>>     Template Record, or a subsequent export time. Likewise, a Template
>>>>>     Withdrawal is only in effect for IPFIX Messages with the same Export
>>>>>     Time as the Template Withdrawal, or a subsequent Export Time.
>>> Perhaps something like...
>>>
>>>    Put another way, a Template only describes Data Records contained
>>>    in IPFIX Messages with the an Export Time greater than or equal to
>>>    the Export Time of the IPFIX Message containing Template Record.
>>>    Or until a Template Withdrawal is received. Likewise, a Template
>>>    Withdrawal is only in effect for IPFIX Messages with the an Export
>>>    Time greater than or equal to the Export Time of the IPFIX Message
>>>    containing Template Withdrawal.  Or until a new Template with the
>>>    withdrawn Template ID is received.
>>>
>>> -Andrew
>>> _______________________________________________
>>> IPFIX mailing list
>>> [email protected]
>>> https://www.ietf.org/mailman/listinfo/ipfix
>> _______________________________________________
>> IPFIX mailing list
>> [email protected]
>> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix