Re: [IPFIX] template lifetime

hi, Paul, all,

On Sep 5, 2011, at 10:53 AM, Paul Aitken wrote:

> Brian,
> 
>>>>> How does a collector determine the correct lifetime to associate with each Template, and how does it know "the Template refresh timeout configured on the Exporting Process." ?
>>>> Solution 1. Out of band configuration with http://tools.ietf.org/html/draft-ietf-ipfix-configuration-model-10. Note: I still envision that IPFIX caches might be configured for a short period of time on IPFIX exporters, for troubleshooting reasons.
>>> This is not mentioned anywhere in IPFIX. It's a hole.
>> Is the intention to plug this in 5101bis? The questionable (or at least, not yet demonstrated) interoperability of the existing spec may give us an opening to do so...
> 
> That's rather the point of this thread.

Good. Now I'm caught up. :)

>>>> Solution 2. CP observes 10 (or 20, or whatever number) template refresh and conclude the right value
>>> Packet loss and network jitter make this problematic.
>> Indeed, but assuming a network that is working _well enough_ that the human in charge doesn't reprovision it due to unacceptable export efficiency, the long term average of template refresh times should approximate an upper bound reasonably well...
> 
> There may not be any such thing as a "long term" average, since templates may come and go at unpredictable times. It may be especially difficult to make any predication about the lifetime of short-lived templates or frequently redefined templates.
> 
> Also, when a template is redefined (ie, same template ID, but different fields or different field order), the lifetime calculation has to be re-started. Clearly a full discussion is missing and is needed if this method is expected to be used.

Ah. Good point... So let's move this down the list for now on account of "effort required to do it right" (indeed, it seems on the face of it that it is harder to do right than the current method, which IMO is impossible to do right, so...)

> 
>>>> Solution 3: the CP doesn't bother and take a very high value
>>> That seems reasonable. Why not specify this as the default behaviour?
>> Infinity seems high enough. :) Coupled with automatic replacement on receipt of a different template with the same ID (keyed by template validity tied to the export time in the message in which it was received), it would seem to solve the synchronization problem.
> 
> This could work. However, we have to recognise the disadvantage of possibly incorrectly decoding some data records where a template redefinition has been missed. A finite lifetime would ensure the records had to be discarded.

I'm still a fan of not even _trying_ to pretend that UDP over a MAC layer that isn't doing some sort of congestion control (even implicit congestion control through "being dedicated and insanely overdimensioned") is reliable enough for every template management corner case (this, indeed, is why IESG pushback put a MUST on IPFIX over SCTP)... If you are going to flow export over UDP, you need to have administrators on _both_ sides of the IPFIX session who know what they're doing, because in limited circumstances things _will_ break, full stop. If you need it to work, don't use the optional protocol, use the mandatory one, where these problems are solved.

> 
>>>>> What are the default and acceptable range of lifetime values?
>>>>> 
>>>>> Should we have a mechanism which allows an Exporting Process to report Template lifetimes to the Collecting Process?
>>>>> eg, by exporting an option of { scope = templateId, lifetime = lifeTimeUnits }, where lifeTimeUnits = u32 milliseconds - which allows 1ms to 49.7 days, with 0 = infinite?
>>>> This is solution 4.
>>> Then it needs to be written in the protocol so EPs export it and CPs expect and action it correctly.
>> If we go with option 4 (unsurprisingly, I'm not a fan), then I would suggest _explicitly_ tying these timeouts to the export time in the message header, not any-old-random-clock the CP happens to have hanging around.
> 
> Clearly there are several ways of solving the problem. I don't really mind which method is used, as long as it fills what seems to be a hole in IPFIX.

On that point we are emphatically in agreement.

Cheers,

Brian
_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix