Re: [IPFIX] review of draft-ietf-ipfix-mediation-protocol-04

hi, Paul, all,

Catching back up on this draft after being away a bit. Replies to randomly selected points inline.

On Jun 28, 2013, at 12:02 AM, Paul Aitken <[email protected]> wrote:

> Benoit,
> 
>> Finally, I took the time to reply to your email.
>> As always, very thorough review
> 
> Thanks. I'm happy with all your feedback, except for the following points:
> 
> 
>>>>    Original Exporter:   An Original Exporter is an IPFIX Device that
>>>>       hosts the Observation Points where the metered IP packets are
>>>>       observed.
>>>> 
>>>>    Original Observation Point:   An Observation Point of the Original
>>>>       Exporter.  In the case of the Intermediate Aggregation Process on
>>>> 
>>> 
>>> These definitions imply that OPs belong to the exporter, which is not the case; they belong to the MP or to the IPFIX device.
>> Changed to 
>> Original Observation Point:   An Observation Point on 
>> the Original
>>       Exporter.  In the case of the Intermediate Aggregation Process on
>> 
> 
> Looking at the OP and OD definitions in 5101 and 5101bis, I see nothing to suggest that OPs are in any way related to Exporters.

Yes, but from the point of view of a Mediator, the Original Observation Point is defined in terms of its relationship to the Original Exporter, no? So this seems fine.

>>> So are we saying that OD zero has a special meaning? Should existing exporters avoid using OD zero in order to avoid confusing Collectors?
>> It's already taken care of by RFC5101bis and RFC5101 btw.
>>  Observation Domain ID
>> 
>>       A 32-bit identifier of the Observation Domain that is locally
>>       unique to the Exporting Process.  The Exporting Process uses the
>>       Observation Domain ID to uniquely identify to the Collecting
>>       Process the Observation Domain that metered the Flows.  It is
>>       RECOMMENDED that this identifier also be unique per IPFIX Device.
>>       Collecting Processes SHOULD use the Transport Session and the
>>       Observation Domain ID field to separate different export streams
>>       originating from the same Exporter. 
>>  The Observation Domain ID
>>       SHOULD be 0 when no specific Observation Domain ID is relevant for
>>       the entire IPFIX Message, for example, when exporting the
>>       Exporting Process Statistics, or in case of a hierarchy of
>>       Collectors when aggregated Data Records are exported.
>> 
>> Now, the only change is SHOULD -> MUST
> 
> You avoided the second part of the question: Should existing exporters avoid using OD zero in order to avoid confusing Collectors?
> 
> I could also ask, "MUST existing exporters avoid using OD zero ..." ?
> 
> It's possibly not directly relevant for this draft, but definitely for 5101bis.

What's the suggested change to 5101bis? There's no particular guidance to avoid OD zero in cases where the Message doesn't contain data for multiple source ODs, but IIRC we left this (and indeed, everything we could possibly could about ODs) open, because an OD is kind of an opaque thing up to the implementor, and only touches the protocol in that it scopes templates and options.

>>>> Claise, et al.           Expires August 29, 2013               [Page 22]
>>>> Internet-Draft               IPFIX MED-PROTO               February 2013
>>>> 
>>>> 
>>>>    | ignoredRecordTotalCount | The total number of Data Records        |
>>>>    |                         | received but not processed by the       |
>>>>    |                         | Intermediate Process.                   |
>>>>    | time first record       | The timestamp of the first record that  |
>>>>    | ignored                 | was ignored by the Intermediate         |
>>>>    |                         | Process.  For Data Records containing   |
>>>>    |                         | timestamp ranges, this SHOULD be taken  |
>>>>    |                         | from the start timestamp of the range;  |
>>>>    |                         | for data records containing no timing   |
>>>>    |                         | information, this SHOULD be taken from  |
>>>>    |                         | the Export Time in the message header   |
>>>>    |                         | of 
>>>> the containing IPFIX Message.  For   |
>>> 
>>> Is this the incoming IPFIX Message? So the Intermediate Process has to examine each incoming Message in some detail, even though it's ignoring them?
>> Yes.
> 
> Well that's just daft. So this is a new definition of "ignoring" which actually means "examining them in detail"? Have you considered a career in politics?

Yes, but they won't let me run for anything here until I get my passport. ;)

And this depends completely on where the bottleneck is. Consider the case where per-record processing is far more expensive than message deframing, in that case, it knows exactly where in the record stream it starts having problems, and can count dropped records because it's operating on a record stream instead of a message stream. It can even keep deframing messages when it knows that it'll have to drop records, because the marginal cost of deframing a message is zero.

If the problem is keeping up with messages, though, yes, you're right, this doesn't work. See below on the tradition of wild guessing in metameasurement.

>>>>    |                         | this timestamp, any of the following    |
>>>>    |                         | timestamp can be used:                  |
>>>>    |                         | observationTimeSeconds,                 |
>>>>    |                         | observationTimeMilliseconds,            |
>>>>    |                         | observationTimeMicroseconds, or         |
>>>>    |                         | observationTimeNanoseconds.             |
>>>>    | time last record        | The timestamp of the last record that   |
>>>>    | ignored                 | was ignored by the Intermediate         |
>>>>    |                         | Process.  For Data Records containing   |
>>>>    |                         | timestamp ranges, this SHOULD be taken  |
>>>>    |                         | from the end timestamp of the range;    |
>>>>    |                         | for data records containing no timing   |
>>>>    |                         | information, this SHOULD be taken from  |
>>>>    |                         | the Export Time in the message header   |
>>>>    |                         | of the containing IPFIX Message.  For   |
>>>>    |                         | this timestamp, any of the following    |
>>>>    |                         | timestamp can be used:                  |
>>>>    |                         | observationTimeSeconds,                 |
>>>>    |                         | observationTimeMilliseconds,            |
>>>>    |                         | observationTimeMicroseconds, or         |
>>>>    |                         | observationTimeNanoseconds.             |
>>>>    +-------------------------+-----------------------------------------+
>>>> 
>>> 
>>> Pleaseaddsomewhitespacetomakethetablereadable.
>> I could not find a way to do it...
> 
> Add some blank lines to vertically separate each definition in the table. Else all the definitions run together.
> 
> ie, make each definition a visually separate chunk. See http://syque.com/cstyle/ch2.8.htm

cf. previous discussions on xml2rfc hacking -- as you've made this comment before on stock xml2rfc tables I've put into IPFIX documents, it may be more productive if you raise this issue with the xml2rfc people instead.

In the meantime we can certainly crudely hack a dividing line in there.

>>>> 10.4.  ignoredRecordTotalCount Information Element
>>>> 
>>>>    Description:   The total number of received Data Records that the
>>>>       Intermediate Process did not process since the (re-)initialization
>>>>       of the Intermediate Process; includes only Data Records not
>>>>       examined or otherwise handled by the Intermediate Process due to
>>>>       resource constraints, not Data Records which were examined or
>>>>       otherwise handled by the Intermediate Process but which merely do
>>>>       not contribute to any exported Data Record due to the operations
>>>>       performed by the Intermediate Process.
>>>> 
>>> 
>>> If a mediator is resource constrained, how can it accurately report this figure?
>> Like in RFC 5101, section 4.2. The Metering Process Reliability Statistics Option Template
>> 
>>   ignoredPacketTotalCount
>>                            The total number of IP packets that the
>>                            Metering Process did not process.
>> 
>>    ignoredOctetTotalCount
>>                            The total number of octets in observed IP
>>                            packets that the Metering Process did not
>>                            process.
>> 
> 
> Sorry, I asked the wrong question. If a mediator is resource constrained, how can it accurately measure this figure?
> 
> It's a similar point to "time first record ignored" above.

It can't accurately measure it at all. But that's not the point.

This is simply an instance of the apparent custom in measurement equipment, all the way up and down the stack, to try to estimate dropped/missing packets/messages/events. Some of the estimates (dropped packet counting in hardware capture) are close enough to accurate that you can use them for debug purposes, if not accounting purposes. Some of them are utter garbage. The quality of the guess is implementation and situation dependent. These counters are provided for consistency in line with this tradition: it's the same situation as with reliability statistics reported by MPs and EPs in 5101(bis), of which I'm also (as a user of data) deeply, deeply skeptical. But one can argue that a low-quality, well-intentioned guess is better than nothing.

Cheers,

Brian
_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix