Re: [IPFIX] TCP flags?

hi Andrew, Paul,

On 23 Jul 2013, at 14:52 , Andrew Feren <[email protected]> wrote:

> Hi Paul,
> 
> On 07/22/2013 03:01 PM, Paul Aitken wrote:
>> Andrew,
>> 
>>> Hi all,
>>> 
>>> I've been on vacation so let me recap and make sure I have all the options straight.
>>> 
>>> Options
>>> 1) add ECE and CWR bits to the current definition with a note that " Collecting Processes should not assume that CWR and ECE were not set simply because they're exported as 0, as previous revisions of the Information Element did not include them".
>>> 
>>> 2) update the current definition to unsigned 16 with similar wording to option 1 if the 8bit reduced size encoding is exported.   Using the extended 16 bits would allow the export of ECN Nonce Sum and remove the ambiguity about CWR and ECE that exists in the 8 bit export.
>>> 
>>> 3) Deprecate current IE and replace it with a new 16 bit IE.
>> 
>> Another possibility was to add a new IE for the uppermost bits. In some cases the two IEs could be exported alongside each other, so the 16 bits would be in order (although that need not always be the case).
> 
> I don't like the option of a new IE for just the high bits.  If we must create a new IE I think we should go all the way and take option 3.  Exporting the an IE for the high bits and the existing IE adjacent to each other is cute, but seems like a lot of complexity for no real gain.

Point.

One thing to consider: we're talking here about the passive observation of ECN, which we hope becomes more interesting as ECN deployment and activation continues, but at the moment is kind of a backwater. While an increasing number of ECN capable hosts exist (per Kuhlewind, Neuner, Trammell, "On the state of ECN and TCP Options in the Internet", PAM 2013, about 30% of the Alexa Top 100k list will happily negotiate ECN with you as of August 2012, and that number is increasing), the servers are defaulting to "ECN on request", but most clients don't request it, so you don't see much ECN actually used (in the same paper, we saw about the same amount of ECN negotiation, passively observed, as we did persistent misuse of the ECT bits in the IP header; since we were using good old-fashioned ("Inf
 lexible"?) NetFlow, we didn't see the ECE or CWR bits here.) One would presume that ECN Nonce only gets used if ECN does, and the ECN Nonce flag doesn't imply calculation of correct nonces, 
 just that ECN Nonce was activated on the flow and the nonce sum was once odd.

> One other question.  With this option would the semantics of the existing IE change if both IEs are exported or would the ECE and CWR bits just remain ambiguous?

One could assume that if an Exporter exports tcpHighControlFlags, that CWR and ECE are also valid. We can make that same assertion about exporting tcpControlFlags as an unsigned16, though.

>> 
>>> Assuming I got the above correct here are my thoughts.
>>> 
>>> If we are going to make the changes for option 1 we might as well go all the way to option 2.  We've covered this before, but having separate data types for different integer lengths is pretty much pointless since the size is always exported in the template.  From a collector stand point I pretty much have to respect the length in the template so increasing this to a unsigned 16 is no real work for me.
>> 
>> What I said before: changing the size doesn't actually change the type. Fundamentally we only have signed and unsigned, in various sizes. If we'd recognised this years ago, we wouldn't need the "reduced size encoding" rule; collectors would just accept whatever the templates tell them.
>> 
>> 
>>> The only real difference I see between options 2 and 3 is that the reduced size encoding for option 2 will always be ambiguous for two bits.  I don't yet have an opinion on how big a deal that is. Can we increase the size of the current TCP flags and deprecate the export of the reduced size encoding?
>> 
>> That could work. However, existing implementations would continue to export 8 bits, where the top 2 bits are ambiguous.
> 
> That is true regardless of what option we ultimately decide on.

An updated IE definition noting this ambiguity in the description, is, I think, necessary (no matter what).

>> 
>> 
>>> I currently have a very slight preference for option 2 (with or without deprecating reduced size encoding), but there is still a quite, but persistent, voice in the back of my head saying option 3 is "the right thing".
>> 
>> Is it possible that some collectors might break if they receive a field with a different size from what 5102 / IANA says? eg, there was a problem with old versions of wireshark where it had built-in expectations of field sizes, and couldn't cope with the change of sampler ID from 8 bits to 16 bits to 32 bits.
> 
> This is a more general issue for wireshark than just sampler ID. Any IE sent with reduced size encoding was liable to blow things up.  8 and 4 byte decodes were implemented for some IEs where people had run into both encodings.  Other sizes would still have caused issues.  I ran into this about a month ago for an IE specified as unsigned64, but with a decode expecting unsigned32.  It appears that the latest dev branch of wireshark is better about this, but I haven't really looked to see how complete the fixes are.

>> If that might be the case, then a new IE is indicated. OTOH, if nobody knows of such a possibility, then (2) might work equally as well.
> 
> I can only speak for my implementation and I don't see 2 as a problem.  However, as you pointed out implementations like wireshark are known to have had issues with similar changes in the past.  That said I'm not sure how much weight I give to wireshark's IPFIX implementation.  Last time I looked wireshark didn't handle other protocol details like template IDs that change after an exporter reboot.

I've got on my list of things to do in my copious free time having a look at improving the wireshark dissector for IPFIX. I mainly use ripfix / python-ipfix for debugging IPFIX because of bad experiences with the wireshark dissector years ago, and I wasn't aware that anyone was trying to use it / that it was still maintained at all.

> I still like option 2, but could be persuaded that 3 is the better option.

Option 2 would be my preferred option; Option 3 (deprecation) seems like a whole lot of procedural effort (especially when at least n EP vendors are already exporting the high two bits regardless of the spec). The tcpHighControlBits option was a hack I proposed in case option 2 was somehow unacceptable.

Cheers,

Brian

>>> On 07/19/2013 10:01 AM, Paul Aitken wrote:
>>>> Brian,
>>>> 
>>>> So we have at least 3 solutions. We need a wider audience and feedback from collector vendors.
>>>> 
>>>> P.
>>>> 
>>>> 
>>>> On 19/07/13 14:51, Brian Trammell wrote:
>>>>> Hi Paul,
>>>>> 
>>>>> Inline, sent from my iPhone
>>>>> 
>>>>> On 19.07.2013, at 15:26, Paul Aitken <[email protected]> wrote:
>>>>> 
>>>>>> Brian,
>>>>>> 
>>>>>>> hi Paul,
>>>>>>> 
>>>>>>> another idea inline.
>>>>>>> 
>>>>>>> On 19 Jul 2013, at 14:48 , Paul Aitken <[email protected]> wrote:
>>>>>>>>>> Q2: how should the ECN Nonce Sum be reported?
>>>>>>>>> My suggestion would be to define this field as an unsigned16:
>>>>>>>>> 
>>>>>>>>> MSb LSb
>>>>>>>>> +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>>>>>>>>    |                           | N | C | E | U | A | P | R | S | F |
>>>>>>>>>    |         Reserved          | S | W | C | R | C | S | S | Y | I |
>>>>>>>>>    |                           |   | R | E | G | K | H | T | N | N |
>>>>>>>>> +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
>>>>>>>>> 
>>>>>>>>> with a specific note that when the IE is encoded as an unsigned8 using reduced-length encoding, it has the following layout:
>>>>>>>>> 
>>>>>>>>>     MSb                         LSb
>>>>>>>>>    +---+---+---+---+---+---+---+---+
>>>>>>>>>    | C | E | U | A | P | R | S | F |
>>>>>>>>>    | W | C | R | C | S | S | Y | I |
>>>>>>>>>    | R | E | G | K | H | T | N | N |
>>>>>>>>>    +---+---+---+---+---+---+---+---+
>>>>>>>>> 
>>>>>>>>> and a further specific note that Collecting Processes should not assume that CWR and ECE were not set simply because they're exported as 0, as previous revisions of the Information Element did not include them.
>>>>>>>>> 
>>>>>>>>> Yes, it's kludgy, but it has the advantage of describing the reality of the status quo.
>>>>>>>> I'm happy with that, except for enlarging the size from u8 to u16. Is that acceptable to collectors?
>>>>>>> As far as I can tell, that's the only open question.
>>>>>>> 
>>>>>>> Another possibility: expand this one to the full 8 bits (with a note about CWR and ECE being potentially unsupported by old EPs), and define a new unsigned8 IE for the high four bits of the increasingly inaccurately named TCP flags byte, to futureproof against use of those three bits.
>>>>>>> 
>>>>>>> This has the advantage of having the same record encoding as the unsigned16 version (if you follow tcpHighControlFlags with tcpControlFlags in the template) at the expense of 4 extra template bytes, while not having any possibility to break collectors that aren't expecting it.
>>>>>> Yes, that would work too.
>>>>>> 
>>>>>> It'd be slightly more efficient to define the new field as u16 containing all the bits, and export either/or.
>>>>>> 
>>>>>> A collector update is required either way...
>>>>> Not if an exporter that's ECN Nonce aware is exporting to a collector that isn't; in that case, it gets the low 8 bits from the tcpControlBits and the ignores the high four (one) as an unknown IE.
>>>>> 
>>>>> Cheers B
>>>> 
>>>> _______________________________________________
>>>> IPFIX mailing list
>>>> [email protected]
>>>> https://www.ietf.org/mailman/listinfo/ipfix
>> 

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix