Re: [IPFIX] basicList clarification
Andrew Feren <[email protected]> Wed, 16 Dec 2015 22:41:05 +0000
| Newsgroups | gmane.ietf.ipfix |
|---|---|
| Message-ID | <[email protected]> |
--===============4788669504545049279== Content-Language: en-US Content-Type: multipart/alternative; boundary="_000_8E7542283B89BB4DB672EB49CEE5AAB7595C1F74PLXRDC01plxrloc_" --_000_8E7542283B89BB4DB672EB49CEE5AAB7595C1F74PLXRDC01plxrloc_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Paul and Gerald, I have slightly mixed feelings on this, but my current bias is towards not = requiring a second IE. I plan to look at this a bit more closely, but here= are a few thoughts. see [ACF] inline... ________________________________ From: IPFIX [[email protected]] on behalf of Paul Aitken [pjaitken@gma= il.com] Sent: Tuesday, December 15, 2015 6:24 PM To: Gerald Naveen A Cc: [email protected] Subject: Re: [IPFIX] basicList clarification Gerald, Hi Paul, Thanks for the responses. It clarifies for most part; while I would also li= ke to clarify few things on my question. (tagged [Gerald] inline) See [PJ] inline... So it appears we would need to define Y as IE and mark X as basicList of Y. What you're proposing is possible if there's a good reason. However we want= to avoid unnecessary duplication of existing IEs into new "list of" IEs as= much as possible, so you should only create X if there's a strong justific= ation (eg it models an existing object). Thanks again On Tue, Dec 15, 2015 at 6:44 PM, Paul Aitken <[email protected]<mailto:pja= [email protected]>> wrote: Gerald, Hi all, It is my understanding from the RFCs that basicList type is an encoding for= list of IPFIX Information Elements. Yes, where the list contains zero or more instances of a single IE. To create the template much more strongly typed, we have defined a custom (= enterprise) Information Element with a new ID (say X), which is defined as = a basicList of specific items (eg., InterfaceName). My question to you all is: Is it mandatory that we have a separate informat= ion element called InterfaceName (say ID Y) and define X as a basicList of = Y ? Yes, that's how the protocol is defined. For our need, it is enough for us, that we define X as a basicList of strin= g (as X is already strongly defined). But string is a type, not info elemen= t. We would like to retain X (instead of defining a template directly with a s= tandard basicList). And we don't have any other need to define a new Inform= ation Element Y. Y defines the list elements which tells the Collector that the list contain= s strings, and it defines the semantics of those strings - eg whether they = contain interface names, user names, a list of selectorNames, or a list of = Application names. Then a basicList of Y tells the Collector that you're ex= porting zero or more of them. [Gerald] This is definitely a choice. However, as I mentioned, this makes t= he template weakly typed (ie., the element is just #291 basicList). We woul= d like to enforce stronger types at the template itself (this was the motiv= e behind defining X) [PJ] This was an objection from the Collector side when we were writing the= IPFIX Structured Data draft, because the collector can no longer tell what= information it's going to receive ahead of time (just "a list"); it has to= check each Data Record to know what information the list contains. On the = other hand, since the basicList is essentially a wrapper around any other I= E, it's not necessary to create new strongly-defined list IEs for each of t= he existing (and potentially all future) IEs. [ACF] From the collector side I still prefer strongly typed templates. The= re may be some middle ground possible like an option template that reports = the range of expected values for a template with a basicList (or other gene= ric structure). I started a draft on this a while back, but it got back bu= rnered. Might be time to dust it off again. Perhaps you should define Y as a single instance of X (so Y is well defined= ), then redefine X as a basicList of Y. Or if you don't want/need to define= both X and Y, then use the basicList IE #291 to export a basicList of (str= ongly defined) Y. [Gerald] Yes, this is also a choice; but we don't need Y for any other reas= on except to fit into the basicList detail. So wanted to know if that can b= e avoided. I got my answer: No (if strongly typed templates). [PJ] Strongly typed templates would require a new "list of" IE for every ex= isting (and future) IE that was to become a list, which is incredibly waste= ful. [ACF] I think there can be different considerations for lists in standard = space and list IEs with a PEN. I realize that the basicList encoding has a information-element ID of the i= tem in payload -- is this mandatory? Yes. The encoding would not be interoperable without it. [PJ] Here I understood the question to be whether the IE ID field was requi= red in the basicList encoding - ie, it seemed that you were proposing a new= encoding where, since X is already strong defined as a basicList of Y, the= re's no need to include Y in the export. [ACF] I think the quibble here is whether Y needs an IE if it is only used = as a list. If an IE already exists you can use it. Requiring Y becomes wa= steful (to borrow your earlier word) in a different way. Assume there are = many lists of A, B, C, D and X. There isn't a need to ever export A, B, C,= D or X as any thing other than a list. The second IE isn't really needed. Or for a strongly typed item X, we could expect the IPFIX parser to be awar= e of contents (interface name) by contract? No. Every collector which implements RFC 6313 (IPFIX Structured Data) shoul= d be able to decode a basicList of Y. However, the mechanism which you prop= ose requires the collector to have special knowledge of the internals of X,= which is not scalable. In short, such an Exporter would not be interoperab= le with all Collectors. [Gerald] X is not a different encoding. X is a basicList (sort of more stro= ngly typed. ie., X is a basicList of Y). So the encoding is just the same a= s #291 standard basicList. But when the template declares a element as X, i= t is clearly typed even without having to look into the data (or it also do= esn't let the exporter send X of Z on the same template). [PJ] What you're proposing is possible and may be useful in some circumstan= ces. For example, for the MIB export draft, we've defined new IEs (#443, #4= 44) which are subTemplateLists, in order to give strong typing information = because the IE contains a row from a table. (See https://tools.ietf.org/html/draft-ietf-ipfix-mib-variable-export-10 an= d http://www.iana.org/assignments/ipfix/ipfix.xhtml) [ACF] A different compromise might be some generic IEs, for exampe, a strin= g IE. Then you could have a list of aNames or bNames. Each of which put t= he (currently non existent) string IE ID as the ID in the strongly typed li= st. Of course that opens the door to generic basicList of generic strings.= Having a single list encoding is nice, but leaving out the ID probably is= n't any worse than fixed length vs var length strings. I reserve the right to change my opinions as I reread 6313, -Andrew P. Pls let me know if my question is unclear. Thanks in advance. - Gerald Naveen The custom IE which you've defined must contain meta-data about the list su= ch as an element count, element lengths, delimiters or separators. It creat= es a new way of encoding list information which the collector must also und= erstand in addtion to the basicList encoding, which is not useful. In short= , you should not create list type elements; instead, use the basicList mech= anism. [Gerald] Agree. We aren't trying to redefine basicList encoding for sure. _______________________________________________ IPFIX mailing list [email protected]<mailto:[email protected]> https://www.ietf.org/mailman/listinfo/ipfix --_000_8E7542283B89BB4DB672EB49CEE5AAB7595C1F74PLXRDC01plxrloc_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <html dir=3D"ltr"> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Diso-8859-= 1"> <style id=3D"owaParaStyle" type=3D"text/css">P {margin-top:0;margin-bottom:= 0;}</style> </head> <body ocsi=3D"0" fpstyle=3D"1" bgcolor=3D"#FFFFFF"> <div style=3D"direction: ltr;font-family: Tahoma;color: #000000;font-size: = 10pt;">Hi Paul and Gerald,<br> <br> I have slightly mixed feelings on this, but my current bias is towards not = requiring a second IE. I plan to look at this a bit more closely, but= here are a few thoughts.<br> <br> see [ACF] inline...<br> <br> <br> <div style=3D"font-family: Times New Roman; color: #000000; font-size: 16px= "> <hr tabindex=3D"-1"> <div style=3D"direction: ltr;" id=3D"divRpF332133"><font size=3D"2" color= =3D"#000000" face=3D"Tahoma"><b>From:</b> IPFIX [[email protected]] on= behalf of Paul Aitken [[email protected]]<br> <b>Sent:</b> Tuesday, December 15, 2015 6:24 PM<br> <b>To:</b> Gerald Naveen A<br> <b>Cc:</b> [email protected]<br> <b>Subject:</b> Re: [IPFIX] basicList clarification<br> </font><br> </div> <div></div> <div> <div class=3D"moz-cite-prefix">Gerald,<br> <br> </div> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div>Hi Paul,<br> <br> </div> Thanks for the responses. It clarifies for most part; while I would also li= ke to clarify few things on my question. (tagged [Gerald] inline)<br> </div> </div> </blockquote> <br> See [PJ] inline...<br> <br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr">So it appears we would need to define Y as IE and mark X a= s basicList of Y.<br> </div> </blockquote> <br> What you're proposing is possible if there's a good reason. However we want= to avoid unnecessary duplication of existing IEs into new "list of&qu= ot; IEs as much as possible, so you should only create X if there's a stron= g justification (eg it models an existing object).<br> <br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div class=3D"gmail_extra">Thanks again<br> <br> </div> <div class=3D"gmail_extra"> <div class=3D"gmail_quote">On Tue, Dec 15, 2015 at 6:44 PM, Paul Aitken <sp= an dir=3D"ltr"> <<a href=3D"mailto:[email protected]" target=3D"_blank">pjaitken@gmail.= com</a>></span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=0A= .8ex; border-left:1px #ccc solid; padding-left:1ex"> <div bgcolor=3D"#FFFFFF">Gerald,<span class=3D""><br> <div><br> </div> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div> <div> <div> <div> <div>Hi all,<br> <br> </div> It is my understanding from the RFCs that basicList type is an encoding for= list of IPFIX Information Elements.<br> </div> </div> </div> </div> </div> </div> </blockquote> <br> </span>Yes, where the list contains zero or more instances of a single IE.<= span class=3D""><br> <br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div> <div> <div>To create the template much more strongly typed, we have defined a cus= tom (enterprise) Information Element with a new ID (say X), which is define= d as a basicList of specific items (eg., InterfaceName).<br> <br> </div> My question to you all is: Is it mandatory that we have a separate informat= ion element called InterfaceName (say ID Y) and define X as a basicList of = Y ?</div> </div> </div> </div> </blockquote> <br> </span>Yes, that's how the protocol is defined.<span class=3D""><br> <br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div> <div>For our need, it is enough for us, that we define X as a basicList of = string (as X is already strongly defined). But string is a type, not info e= lement.<br> <br> </div> We would like to retain X (instead of defining a template directly with a s= tandard basicList). And we don't have any other need to define a new Inform= ation Element Y.<br> </div> </div> </div> </blockquote> <br> </span>Y defines the list elements which tells the Collector that the list = contains strings, and it defines the semantics of those strings - eg whethe= r they contain interface names, user names, a list of selectorNames, or a l= ist of Application names. Then a basicList of Y tells the Collector that you're exporting zero or more of t= hem.<br> </div> </blockquote> <div>[Gerald] This is definitely a choice. However, as I mentioned, this ma= kes the template weakly typed (ie., the element is just #291 basicList). We= would like to enforce stronger types at the template itself (this was the = motive behind defining X) <br> </div> </div> </div> </div> </div> </blockquote> <br> [PJ] This was an objection from the Collector side when we were writing the= IPFIX Structured Data draft, because the collector can no longer tell what= information it's going to receive ahead of time (just "a list");= it has to check each Data Record to know what information the list contains. On the other hand, since the basicList= is essentially a wrapper around any other IE, it's not necessary to create= new strongly-defined list IEs for each of the existing (and potentially al= l future) IEs.<br> <br> [ACF] From the collector side I still prefer strongly typed templates. = ; There may be some middle ground possible like an option template that rep= orts the range of expected values for a template with a basicList (or other= generic structure). I started a draft on this a while back, but it got back burnered. Might be time = to dust it off again.<br> <br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=0A= .8ex; border-left:1px #ccc solid; padding-left:1ex"> <div bgcolor=3D"#FFFFFF">Perhaps you should define Y as a single instance o= f X (so Y is well defined), then redefine X as a basicList of Y. Or if you = don't want/need to define both X and Y, then use the basicList IE #291 to e= xport a basicList of (strongly defined) Y.<span class=3D""><br> </span></div> </blockquote> <div>[Gerald] Yes, this is also a choice; but we don't need Y for any other= reason except to fit into the basicList detail. So wanted to know if that = can be avoided. I got my answer: No (if strongly typed templates).<br> </div> </div> </div> </div> </div> </blockquote> <br> [PJ] Strongly typed templates would require a new "list of" IE fo= r every existing (and future) IE that was to become a list, which is incred= ibly wasteful. <br> <br> [ACF] I think there can be different considerations for lists in stan= dard space and list IEs with a PEN.<br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=0A= .8ex; border-left:1px #ccc solid; padding-left:1ex"> <div bgcolor=3D"#FFFFFF"><span class=3D""> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div>I realize that the basicList encoding has a information-element ID of = the item in payload -- is this mandatory?</div> </div> </div> </blockquote> <br> </span>Yes. The encoding would not be interoperable without it.<span class= =3D""><br> </span></div> </blockquote> </div> </div> </div> </div> </blockquote> <br> [PJ] Here I understood the question to be whether the IE ID field was requi= red in the basicList encoding - ie, it seemed that you were proposing a new= encoding where, since X is already strong defined as a basicList of Y, the= re's no need to include Y in the export.<br> <br> [ACF] I think the quibble here is whether Y needs an IE if it is only used = as a list. If an IE already exists you can use it. Requiring Y = becomes wasteful (to borrow your earlier word) in a different way. As= sume there are many lists of A, B, C, D and X. There isn't a need to ever export A, B, C, D or X as any thing other than = a list. The second IE isn't really needed.<br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=0A= .8ex; border-left:1px #ccc solid; padding-left:1ex"> <div bgcolor=3D"#FFFFFF"><span class=3D""> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div>Or for a strongly typed item X, we could expect the IPFIX parser to be= aware of contents (interface name) by contract?</div> </div> </div> </blockquote> <br> </span>No. Every collector which implements RFC 6313 (IPFIX Structured Data= ) should be able to decode a basicList of Y. However, the mechanism which y= ou propose requires the collector to have special knowledge of the internal= s of X, which is not scalable. In short, such an Exporter would not be interoperable with all Collectors.<br= > </div> </blockquote> <div>[Gerald] X is not a different encoding. X is a basicList (sort of more= strongly typed. ie., X is a basicList of Y). So the encoding is just the s= ame as #291 standard basicList. But when the template declares a element as= X, it is clearly typed even without having to look into the data (or it also doesn't let the exporter send X o= f Z on the same template).<br> </div> </div> </div> </div> </div> </blockquote> <br> [PJ] What you're proposing is possible and may be useful in some circumstan= ces. For example, for the MIB export draft, we've defined new IEs (#443, #4= 44) which are subTemplateLists, in order to give strong typing information = because the IE contains a row from a table.<br> <br> (See <a class=3D"moz-txt-link-freetext" href=3D"https://tools.ietf.org/html= /draft-ietf-ipfix-mib-variable-export-10" target=3D"_blank"> https://tools.ietf.org/html/draft-ietf-ipfix-mib-variable-export-10</a> and= <a class=3D"moz-txt-link-freetext" href=3D"http://www.iana.org/assignments= /ipfix/ipfix.xhtml" target=3D"_blank"> http://www.iana.org/assignments/ipfix/ipfix.xhtml</a>)<br> <br> [ACF] A different compromise might be some generic IEs, for exampe, a strin= g IE. Then you could have a list of aNames or bNames. Each of w= hich put the (currently non existent) string IE ID as the ID in the strongl= y typed list. Of course that opens the door to generic basicList of generic strings. Having a single list encodi= ng is nice, but leaving out the ID probably isn't any worse than fixed leng= th vs var length strings.<br> <br> I reserve the right to change my opinions as I reread 6313,<br> -Andrew<br> <br> P.<br> <br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div class=3D"gmail_extra"> <div class=3D"gmail_quote"> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=0A= .8ex; border-left:1px #ccc solid; padding-left:1ex"> <div bgcolor=3D"#FFFFFF"><span class=3D""><br> <blockquote type=3D"cite"> <div dir=3D"ltr"> <div> <div>Pls let me know if my question is unclear.<br> </div> <div><br> </div> Thanks in advance.<br> </div> - Gerald Naveen<br> </div> </blockquote> <br> <br> </span>The custom IE which you've defined must contain meta-data about the = list such as an element count, element lengths, delimiters or separators. I= t creates a new way of encoding list information which the collector must a= lso understand in addtion to the basicList encoding, which is not useful. In short, you should not create l= ist type elements; instead, use the basicList mechanism.<br> </div> </blockquote> <div>[Gerald] Agree. We aren't trying to redefine basicList encoding for su= re. <br> </div> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=0A= .8ex; border-left:1px #ccc solid; padding-left:1ex"> <div bgcolor=3D"#FFFFFF"><br> <blockquote type=3D"cite"><br> <fieldset target=3D"_blank"></fieldset> <br> <pre>_______________________________________________=0A= IPFIX mailing list=0A= <a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>=0A= <a href=3D"https://www.ietf.org/mailman/listinfo/ipfix" target=3D"_blank">h= ttps://www.ietf.org/mailman/listinfo/ipfix</a>=0A= </pre> </blockquote> <br> </div> </blockquote> </div> <br> </div> </div> </div> </blockquote> <br> </div> </div> </div> </body> </html> --_000_8E7542283B89BB4DB672EB49CEE5AAB7595C1F74PLXRDC01plxrloc_-- --===============4788669504545049279== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPFIX mailing list [email protected] https://www.ietf.org/mailman/listinfo/ipfix --===============4788669504545049279==--