[IPFIX] draft-aitken-ipfix-unobserved-fields-00

                 Reporting Unobserved Fields in IPFIX 

  
  
  
 Internet-Draft                                             P. Aitken
 draft-aitken-ipfix-unobserved-fields-00.txt            Cisco Systems
 Expires: February, 2009 
  
  
                                                         October 2011
  
  
  
                 Reporting Unobserved Fields in IPFIX 
                draft-aitken-ipfix-unobserved-fields-00 
     
  
    Status of this Memo 
     
    By submitting this Internet-Draft, each author represents that 
    any applicable patent or other IPR claims of which he or she is 
    aware have been or will be disclosed, and any of which he or she 
    becomes aware will be disclosed, in accordance with Section 6 of 
    BCP 79. 
     
    Internet-Drafts are working documents of the Internet 
    Engineering Task Force (IETF), its areas, and its working 
    groups.  Note that other groups may also distribute working 
    documents as Internet-Drafts.  
     
    Internet-Drafts are draft documents valid for a maximum of six 
    months and may be updated, replaced, or obsoleted by other 
    documents at any time.  It is inappropriate to use 
    Internet-Drafts as reference material or to cite them other than 
    as "work in progress."  
     
    The list of current Internet-Drafts can be accessed at 
    http://www.ietf.org/1id-abstracts.txt. 
     
    The list of Internet-Draft Shadow Directories can be accessed at  
    http://www.ietf.org/shadow.html. 
     
    This Internet-Draft will expire on May 2007. 
     
  
     
    Copyright Notice  
     
    Copyright (C) The Internet Society (2006). 
  

















                                                        
 Aitken                   Expires Aug 2011                  [Page 1] 

  
 Abstract 
     
    This document discusses several methods to report unobserved 
    fields in IPFIX and the advantages and disadvantage of each. 
     
     
 Conventions used in this document 
  
     The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 
     NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and 
     "OPTIONAL" in this document are to be interpreted as described 
     in RFC 2119 [RFC2119]. 
      
      
 Table of Contents 
  
    1.   Introduction................................................3
    2.   Terminology.................................................3
    3.   Possibilities...............................................3
    3.1    Multiple templates........................................3
    3.2    CommonProperties..........................................4
    3.3    Default Values............................................4 
    3.3.1    Default of Zero.........................................5 
    3.3.2    Field-specific default values...........................5 
    3.4    "Observed Fields" Bitfield................................5
    3.5    Length of Zero............................................6
    4.   Conclusion..................................................6 
    5.   Security Considerations.....................................6 
    6.   IANA Considerations.........................................6 
    7.   References..................................................6 
    7.1    Normative References......................................6 
    7.2    Informative References....................................6 
    8.   Acknowledgements............................................7
    9.   Author's Addresses..........................................7
    10.  Intellectual Property Statement.............................7
    11.  Copyright Statement.........................................7
    12.  Disclaimer..................................................8
     
     
  

























  

  
 1. Introduction 
     
    The IPFIX information model [RFC5102] contains a wide variety of 
    fields [IANA-IPFIX], which are not always present in all 
    traffic. For example, ICMP type and code. Indeed, some fields 
    are mutually exclusive. For example, IPv4 / IPv6 address fields; 
    UDP / TCP port numbers. 
     
    When an IPFIX Metering Process monitors a single such field, it 
    simply reports whenever appropriate traffic is seen. ie, 
    whenever the Observed Traffic Stream [RFC5476] contains the 
    field to be metered. No output is generated whenever the 
    monitored field is not present. 
     
    The IPFIX protocol lacks a method for the Metering Process to 
    actively express that although fields were being monitored, no 
    relevant observations were made. Therefore the Collecting 
    Process cannot know whether the Metering Process was actively 
    monitoring the field, and can only infer from the lack of export 
    that no relevant observations were made. 
     
    Further, when a Metering Process monitors a combination of 
    fields, some may be present while others are not. Therefore the 
    Metering Process observes values for some fields, though not for 
    others. 
     
    The IPFIX Protocol [RFC5101] requires the Exporting Process to 
    employ a number of templates in order to export these 
    combinations, using one template for each observed combination 
    of fields. Since the number of templates required grows 
    exponentially with the number of mutually-exclusive fields, the 
    templates can quickly become unmanageable. Indeed, just a few 
    sets of mutually exclusive fields is sufficient to exhaust the 
    template number space. 
     
    Clearly the IPFIX protocol [RFC5101] would benefit from a method 
    for the Metering Process to actively express that although 
    fields were being monitored, no relevant observations were made. 
     
    This draft discusses various options for reporting such 
    unobserved fields, and the advantages and disadvantages of each, 
    with the ultimate goal of recommending and specify one method as 
    an extension to the IPFIX Protocol [RFC5101]. 


 2. Terminology 
  
    The terminology in this draft is fully aligned with the IPFIX 
    terminology, per section 2 of [RFC5101]. 
     
    Unobserved Field: a field which a Metering Process is metering, 
    but for which no traffic has been seen or no data is available.


 3. Possibilities 
     
    This section discusses various possibilities for unobserved 
    fields. 

  3.1 Multiple templates 
     
    The NetFlow v9 [RFC3954] and IPFIX protocol [RFC5101] are both 
    template based. Templates express which fields are present in 
    the exported Data Records. 
     
  

  
    When no value has been observed for a particular field, a new 
    template is generated without that field. Thus no value is 
    exported for the unobserved field. 
     
    While this prevents an incorrect or misleading value from being 
    exported for the field, it may require a great many Templates to 
    be created. The number may soon become unmanageable, or may 
    exceed the Template number space. 
     
    TODO: example. 
     
    The main issue with this solution is that no data is exported 
    about unobserved fields - so the Collecting Process cannot tell 
    whether the field was not being observed by the Metering 
    Process, or was being observed but no relevant traffic was seen. 
     
  3.2 CommonProperties 
     
    This solution divides Data Records into a core part in which the 
    fields are always observed, and additional parts according to 
    which other fields are observed. These are exported using the 
    method described in [RFC5473]. 
     
    One template is required to express which fields are present in 
    the core, and one Template is required for each combination of 
    additional fields. Since multiple templates are required, the 
    number of Templates may soon become unmanageable or may exceed 
    the Template number space. 
     
    The main issue with this solution is that again, like the 
    Multiple Template case, no data is exported about unobserved 
    fields - so the Collecting Process cannot tell whether the field 
    was not being observed, or was being observed but no relevant 
    traffic was seen. 
     
    TODO: does CP work for NFv9? 
     
  3.3 Default Values 
     
    With this solution, a single Template specifies all the fields 
    which the Metering Process was asked to observe. Fields for 
    which no value was observed, or for which the value is 
    unavailable, are exported with a default value. 
     
    Note that there are two cases: 
     
       1. Unavailable / Not applicable: 
          
         The monitored field was not present in, or not applicable 
         to, the observed traffic. 
         eg, when RTP SSRC is requested for a TCP flow. 
  
       2. Not Calculated: 
        
         Although the required fields exist, something is preventing 
         the calculation from occurring. For example, not enough 
         data has been collected to calculate a TCP round-trip time.  
     
    Note that multiple default values are required only if it's 
    necessary to differentiate these cases. In general, both cases 
    are represented by a single value. 
     
    Since only one template is used, this scheme is trivial to 
    implement and works for both NFv9 and IPFIX. 
     
  

  
    The default value can be one of two kinds as discussed below. 
     
 3.3.1   Default of Zero 
  
    All unobserved fields are exported with the value zero. Neither 
    the Exporting Process nor the Collecting Process needs any extra 
    knowledge about the field. 
     
    However, if zero is a valid value for the field, it will be 
    impossible to distinguish an unobserved field from a real 
    observation. Eg, when reporting the number of lost packets, 
    packetsLost = 0 seems to indicate that no traffic was lost, when 
    it may be intended to indicate that there is no relevant 
    information to report. 
  
 3.3.2   Field-specific default values 
  
    Each field is provided with a special "unobserved" value, which 
    is outside the normal range of observed values. 
     
    When no value is observed for the field, it's exported with the 
    "unobserved" value. This value varies from field to field. 
     
    Eg, when reporting that traffic lost is not relevant to the 
    current flow, packetsLost may be set to 65535 or -1. 
     
    In this case, both the Exporting Process and the Collecting 
    Process need extra knowledge about each individual field. 
     
    Since the "unobserved" value is outwith the normal range of 
    values for the field, it will be possible to distinguish an 
    unobserved field from a real observation. 
     
    However, not all fields have a suitable value. (TODO: example) 
     
  3.4 "Observed Fields" Bitfield 
     
    With this solution, a single template specifies all the fields 
    which the Metering Process was asked to observe. The template 
    contains an additional bitfield which is exported in the Data 
    Record along with the flow data. This bitfield is similar to the 
    flowKeyIndicator [IANA-IPFIX, #173], in that each bit 
    corresponds to one field in the flow record. Each bit indicates 
    whether or not a value was observed for the corresponding field. 
     
    The Collecting Process examines the bitfield and disregards any 
    unobserved fields. Unobserved fields may therefore be exported 
    with any value, since they will be disregarded. 
     
    Since it uses only one template, this scheme is trivial to 
    implement and works for both NFv9 and IPFIX. 
     
    However, a mediator must understand the bitfield and correctly 
    interpret it. eg, if the mediator is aggregating Data Records, 
    it must pay attention to the bitfield. If fields are added or 
    removed from the Flow Record, bits in the bitfield must be 
    shifted accordingly. Therefore this requires changes to IPFIX 
    record processing. 
     
    Note that if the "Observed Fields" bitfield is sent in an IPFIX 
    Options Record, it expresses which fields are valid in that 
    Options Data Record. It's not possible to use option scoping to 
    report the Observed Fields bitfield for any other Record. 
     

  

  
  3.5 Length of Zero. 
     
    This method exports a single Template which specifies all the 
    fields which the Metering Process was asked to observe. Fields 
    for which no value was observed are exported with a length of 
    zero. Therefore unobserved fields are actively indicated. 
     
    Fields which may be unobserved must be anticipated ahead of time 
    and specified in the Template using IPFIX variable-length 
    encoding. 
     
    While this method only requires a single Template, it doesn't 
    work for NFv9 export since NFv9 doesn't support variable-length 
    encoding. 
     
    It also does not address the not-applicable versus not-
    calculated case discussed in section 3.3, which is needed for 
    some fields. 
     
     
 4. Conclusion. 
     
    Several methods of encoding "unobserved" fields have been 
    presented. Each has pros and cons. 
     
    TODO: weigh up the pros and cons and recommend a solution. 
     
     
 5. Security Considerations 
     
    The same security considerations as for the IPFIX protocol 
    apply. 
     
     
 6. IANA Considerations 
     
    There are no IANA considerations at this time. 
     
    Some of the solutions discussed in section 2 require additional 
    Information Elements to be allocated. 
     
     
 7. References 
                                            
     
  7.1 Normative References 
  
    [RFC5101] Claise, B., Ed., "Specification of the IP Flow 
              Information Export (IPFIX) Protocol for the Exchange 
              of IP Traffic Flow Information", RFC 5101, 
              January 2008. 
     
    [RFC2119] S. Bradner, Key words for use in RFCs to Indicate 
              Requirement Levels, BCP 14, RFC 2119, March 1997 
     
     
  7.2 Informative References 
     
    [RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., 
              and J. Meyer, "Information Model for IP Flow 
              Information Export", RFC 5102, January 2008. 
     



  

  
    [RFC5473] Boschi, E., Mark, L., and B. Claise, "Reducing 
              Redundancy in IP Flow Information Export (IPFIX) and 
              Packet Sampling (PSAMP) Reports", 
              RFC 5473, March 2009. 
     
    [RFC5476] Claise, B., Ed., "Packet Sampling (PSAMP) 
              Protocol Specifications", RFC 5476, March 2009. 
     
    [RFC3954] Claise, B., Ed., "Cisco Systems NetFlow Services 
              Export Version 9", RFC 3954, October 2004. 
     
    [IANA-IPFIX]  http://www.iana.org/assignments/ipfix/ipfix.xml 
     
     
     
 8. Acknowledgements 
     
    Thanks to Aamer Akhter for initial review and feedback. 
     
     
 9. Author's Addresses 
     
        Paul Aitken 
        Cisco Systems (Scotland) Ltd. 
        96 Commercial Quay 
        Commercial Street  
        Edinburgh, EH6 6LX, United Kingdom 
        Phone: +44 131 561 3616 
        Email: [email protected] 
  
             
 10.   Intellectual Property Statement 
     
    The IETF takes no position regarding the validity or scope of 
    any Intellectual Property Rights or other rights that might be 
    claimed to pertain to the implementation or use of the 
    technology described in this document or the extent to which any 
    license under such rights might or might not be available; nor 
    does it represent that it has made any independent effort to 
    identify any such rights.  Information on the procedures with 
    respect to rights in RFC documents can be found in BCP 78 and 
    BCP 79. 
    Copies of IPR disclosures made to the IETF Secretariat and any 
    assurances of licenses to be made available, or the result of an 
    attempt made to obtain a general license or permission for the 
    use of such proprietary rights by implementers or users of this 
    specification can be obtained from the IETF on-line IPR 
    repository at http://www.ietf.org/ipr. 
     
    The IETF invites any interested party to bring to its attention 
    any copyrights, patents or patent applications, or other 
    proprietary rights that may cover technology that may be 
    required to implement this standard. Please address the 
    information to the IETF at [email protected]. 
     
 11.   Copyright Statement 
     
    Copyright (C) The Internet Society (2011). This document is 
    subject to the rights, licenses and restrictions contained in 
    BCP 78, and except as set forth therein, the authors retain all 
    their rights. 
     



  

  
 12.Disclaimer  
     
    This document and the information contained herein are provided 
    on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 
    REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND 
    THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, 
    EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY 
    THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY 
    RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS 
    FOR A PARTICULAR PURPOSE.