Re: [IPFIX] review of draft-trammell-ipfix-a9n-03

Brian,

Herewith feedback on a couple of your responses to my previous feedback.

I'm almost done reviewing -a9n-05.

P.

>>>     Key aggregation   is a spatial aggregation operation which results in
>>>        the addition, modification, or deletion of Flow Key fields in the
>>>        partially aggregated Flows.  New Flow Key fields may be derived
>>>        from existing Flow Key fields (e.g., looking up an AS number for
>>>        an IP address), or "promoted" from non-Key fields (e.g., when
>>>        aggregating Flows by packet count per Flow).  Key aggregation can
>>>
>> The above sounds like any non-key can be promoted, which isn't true and needs to be discussed (see later).
>>
>> Consider, "or derived from certain specific non-Key fields" ?
> Okay... although I'm not convinced there exist non-promoteable fields. Certainly some of these might not make a whole lot of sense sense, but you can always count all incoming flows that have some property equal to each distinct value of that property.

A non-key field holds a single value from a set of possible values. 
There's generally no information about how representative that value is 
of the set: it could be the first or last observation, or it could be 
derived from some operation (min, max, average, OR, most frequent, ...). 
If you wanted to know how representative it is, you'd have observed it 
as a key field.

"Unrepresentative" non-key fields shouldn't be promoted to key fields.

The only exception is when the NK field is representative, ie despite 
being NK it contains the only value which was ever observed for the 
field, so there's no difference between it being NK or key.


>>>     counter distribution is greatly simplified by the choice of an
>>>     interval longer than the duration of longest original Flow, itself
>>>     generally determined by the original Flow's Metering Process active
>>>     timeout; in this case an original Flow can contribute to at most two
>>>     Aggregated Flows, and the more complex value distribution methods
>>>     become inapplicable.
>>>
>> This only considers the Metering Process; it fails to consider any delay in the Exporting Process.
>>
>> eg, consider and implementation with 64K cache entries which are checked for exporting at a rate of 1000 entries/second. It will take>  1 minute to check and export the entire cache.
> And the timestamps are put on the flows by the EP as opposed to the MP?

No, except for the export header timestamp. The point is that the flows 
continue to exist as cache entries for longer than might be expected, so 
traffic from the next interval could be added to flows (cache entries) 
from the previous interval.


>>>     from correlation of the original Flow information with some external
>>>     source.  There are two basic operations here.  First, Aggregated Flow
>>>     Keys may be derived directly from original Flow Keys through
>>>     reduction, or the dropping of fields or precision in the original
>>>     Flow Keys.  Second, an Aggregated Flow Key may be derived through
>>>     replacement, e.g. by removing one or more fields from the original
>>>     Flow and replacing them with a fields derived from the removed
>>>     fields.  Replacement may refer to external information (e.g., IP to
>>>     AS number mappings).  Replacement need not replace only key fields.
>>>     For example, consider an application which aggregates flows by packet
>>>     count (i.e., generating an Aggregated Flow for all one-packet Flows,
>>>     one for all two-packet Flows, and so on).  This application would
>>>     promote the packet count to a Flow Key field.
>>
>> Say more about promotion, ie it can only be done on non-key fields which represent a single unique value. eg, counters, time.
>> If a NK field might be any value from a set of>1 values (eg, an IP address, port number, AS, TOS) then it's not suitable for promotion to Key, because there's no way of knowing which other values were or were not observed, and therefore no way of dividing the flow amongst those values.

> I'm not sure I follow you here. Why does the potential set of values (and its difference to the actual set) affect the promotion ability? Can you give a more detailed example?

See what I said above about how representative the value is. eg, an IP 
address which represents just one of many hundreds of possible values 
can't be promoted from non-key to key since it simply is not a key 
field. If it was a key field, each flow would have been split into many 
hundreds of flows.

However, if that IP address was invariant - eg, you're monitoring all 
the traffic into a server, and it's the destination address of the 
server - then it won't make any difference whether it's observed as key 
or non-key: in the end, the flows will be the same. So if it was 
observed as NK, it could be promoted to key without affecting any of the 
observations.


P.
_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix