Re: [IPFIX] IPFIX observationPointId uniqueness

Hi Paul, Gerhard,

I can't speak for other collectors, but I don't see a problem with
increasing the length of observationPointId.

I never saw any upside to doing anything other than respecting the length
in a template when that length in the template differs from the nominal
type size of the IE.  For reduced size encoding this is a specified
behavior disallowing increased size encoding seemed like extra work with
no return.

Also I think Gerhard made a good case for increasing the size being a
better option than the original proposal.  I didn't have an problem with
the original proposal, but after considering Gerhard's arguments I think
increasing the length of observationPointId is less likely to have any
negative consequences.

-Andrew


On 3/28/13 5:52 AM, "Paul Aitken" <[email protected]> wrote:

>Gerhard,
>
>This question needs input from collector experts on this list.
>
>P.
>
>
>On 27/03/13 20:54, Gerhard Muenz wrote:
>> Paul,
>>
>> I do not see any problem with 64bit OP ID.
>>
>> I think, we had a similar motivation for having 64bit selector
>> IDs/selection sequence IDs.
>>
>> Will it confuse existing collectors? (hopefully not)
>>
>> Regards,
>> Gerhard
>>
>>
>> On 26.03.2013 13:20, Paul Aitken wrote:
>>> Gerhard,
>>>
>>> My problem is mapping multiple discrete 32-bit number spaces into a
>>> single 32-bit Observation Point ID.
>>>
>>> Keeping track of the mapping could be impossibly expensive: in the
>>>worst
>>> case, 2^32 IDs x { (8-bit source type) + (32-bit source ID) + (32-bit
>>> OPID) } = 36G of memory :-(
>>>
>>> This suggests that the number space should be carved, so source 1 uses
>>> OP IDs 0 - 999, source 2 uses OP IDs 1,000 - 1,999 etc. However I can't
>>> do that because there are only 32-bits in the destination (OP ID)
>>>number
>>> space. Each of the source number spaces is also 32-bits; I can't know
>>> whether they'll be linearly allocated or sparsely populated (eg, by a
>>> hashing algorithm), so I can't impose that certain bits (eg the topmost
>>> bits) be used for the carving.
>>>
>>> A 64-bit OP ID would really help me.
>>>
>>> Thanks,
>>> P.
>>>
>>>
>>> On 13/03/13 21:42, Gerhard Muenz wrote:
>>>>
>>>> Paul,
>>>>
>>>> I always thought of OPID, ODID, MPID, EPID etc. as being abstract
>>>> identifiers. If an implementation can map them directly to
>>>> implementation-specific internal parameters, that's fine. If not, a
>>>> mapping needs to be provided by the device. That's the same for other
>>>> data models.
>>>>
>>>> For example, you would also need to assign ipfixObservationPointIndex
>>>> for each OP if you implement IPFIX-MIB. You could use the same value
>>>> for the OPID IE.
>>>>
>>>> If there is no other solution than map 32 bit directly into OPID, I
>>>> would prefer changing OPID to unsigned64.
>>>>
>>>> Regards,
>>>> Gerhard
>>>>
>>>>
>>>> On 13.03.2013 11:26, Paul Aitken wrote:
>>>>> Gerhard,
>>>>>
>>>>> We are monitoring traffic in multiple places - at the ingress and
>>>>> egress
>>>>> interfaces, at processes, in ACLs, in policies.
>>>>>
>>>>> Each of these has a unique 32-bit ID (ie, there may be up to N x
>>>>> 32-bit
>>>>> IDs).
>>>>>
>>>>> However, these IDs are in unique number spaces. eg, the interface ID
>>>>> could happen to be the same as the process ID or the same as the
>>>>> policy
>>>>> ID. So it's not possible to say what the ID represents without
>>>>>knowing
>>>>> the type.
>>>>>
>>>>> You would meet this issue if you monitor interfaces and also want to
>>>>> report PSAMP stats for each selection process. Somehow you need to
>>>>> report distinct IDs for the interfaces versus the processes.
>>>>>
>>>>> Several possibilities come to mind:
>>>>>
>>>>> 1. use multiple different IEs, one per observation point type, with
>>>>> multiple templates. This is not ideal, because we have to request new
>>>>> IEs for each new observation point type, and store / export multiple
>>>>> templates. ie, it adds a layer of unnecessary complexity rather than
>>>>> rather than simplifying the export.
>>>>>
>>>>> 2. implement a conversion function, mapping the N x 32-bit IDs into
>>>>> the
>>>>> 1 x 32-bit observationDomainID space. Obviously only a limited
>>>>> number of
>>>>> conversions (1/N) are possible, and there'll be a trade-off between
>>>>> the
>>>>> table size and lookup time.
>>>>>
>>>>> 3. associate an observationPointType with each observationPointID,
>>>>>and
>>>>> not require any conversion at all. Each internal ID is mapped 1:1
>>>>>with
>>>>> an observationPointId, with uniqueness being determined by the
>>>>> associated observationPointType. This is the simplest and cleanest
>>>>> solution.
>>>>>
>>>>>
>>>>> So to answer your questions directly:
>>>>>
>>>>> - Do you really need OPID for your purpose? Have you thought about
>>>>> using
>>>>> IEs like ingressPhysicalInterface or ingressInterface? Or, if these
>>>>> IEs
>>>>> are not appropriate, about defining new IEs?
>>>>>
>>>>>        - per (1) above, this makes for an unnecessarily complex
>>>>> solution.
>>>>>
>>>>>
>>>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>>>> you thought about assigning the different sources (interface cards,
>>>>> VLANs) to different ODs?
>>>>>
>>>>>        - they are all within a single OD.
>>>>>        - Example use case: report packet counts as traffic progresses
>>>>> through selection and filtering within the router. The report will be
>>>>> exported in a single message with a single OD in the header.
>>>>>            It's not possible to export a single cohesive report using
>>>>> multiple ODs.
>>>>>
>>>>> P.
>>>>>
>>>>>
>>>>>
>>>>> On 15/02/13 09:25, Gerhard Muenz wrote:
>>>>>>
>>>>>> Benoit, Paul,
>>>>>>
>>>>>> Like Benoit, I do not have a good feeling about this proposal. On
>>>>>>the
>>>>>> other hand, I have not found any RFC that really requires the use of
>>>>>> OPIDs. Even RFC5476 leaves it open whether the OP is identified by
>>>>>> OPID or any other IE. Therefore, I have been silent until now.
>>>>>>
>>>>>> IPFIX-CONFIG assumes that the OPID is assigned by the IPFIX device,
>>>>>> i.e. it is not a configuration parameter. I think that we agree
>>>>>>here.
>>>>>>
>>>>>> Up to now, the IPFIX RFCs do not impose any restriction on the OPID
>>>>>> except for the uniqueness per OD. This means that various
>>>>>> implementations are possible right now:
>>>>>> 1) OPID is equal to an internal identifier (e.g. internal interface
>>>>>> number) or a non-IPFIX MIB index (e.g. ifIndex, entPhysicalIndex).
>>>>>> 2) OPID is equal to the IPFIX-MIB index ipfixObservationPointIndex
>>>>>> 3) OPID is unrelated to any other index, the IPFIX device provides
>>>>>> the
>>>>>> internal mapping
>>>>>>
>>>>>> If IPFIX-MIB is supported by the device, the natural choice for me
>>>>>> would be 2). This would allow easy mapping of MIB entries and IPFIX
>>>>>> export.
>>>>>>
>>>>>> As Paul points out, 1) only works as long as the internal
>>>>>>identifiers
>>>>>> are unique per OD.
>>>>>>
>>>>>> My questions to Paul are:
>>>>>> - Do you really need OPID for your purpose? Have you thought about
>>>>>> using IEs like ingressPhysicalInterface or ingressInterface? Or, if
>>>>>> these IEs are not appropriate, about defining new IEs?
>>>>>> - If you need to use OPIDs and if you want to avoid any mapping,
>>>>>>have
>>>>>> you thought about assigning the different sources (interface cards,
>>>>>> VLANs) to different ODs?
>>>>>>
>>>>>> Regards,
>>>>>> Gerhard
>>>>>>
>>>>>>
>>>>>> On 15.02.2013 01:23, Benoit Claise wrote:
>>>>>>> Paul,
>>>>>>>
>>>>>>> At first glance, it looks reasonable.
>>>>>>> But what would be the consequence on the Selection Sequence Report
>>>>>>> Interpretation(see
>>>>>>> http://tools.ietf.org/html/rfc5476#section-6.5.1). If
>>>>>>> we use the observationPointId and it's not unique, then we need to
>>>>>>> include the observationPointType?
>>>>>>> Note that the sentence "It is RECOMMENDED that this identifier is
>>>>>>> also
>>>>>>> unique per IPFIX Device." was inserted specifically for this.
>>>>>>>
>>>>>>> Regards, Benot
>>>>>>>> Dear IPFIX experts,
>>>>>>>>
>>>>>>>> IPFIX observationPointId (#138) is defined as:
>>>>>>>>
>>>>>>>>            An identifier of an Observation Point that is unique
>>>>>>>>per
>>>>>>>>            Observation Domain.  It is RECOMMENDED that this
>>>>>>>> identifier is
>>>>>>>>            also unique per IPFIX Device.  Typically, this
>>>>>>>> Information
>>>>>>>>            Element is used for limiting the scope of other
>>>>>>>> Information
>>>>>>>>            Elements.
>>>>>>>>
>>>>>>>> We now also have observationPointType (#277), defined as:
>>>>>>>>
>>>>>>>>             Type of observation point. Values assigned to date
>>>>>>>>are:
>>>>>>>>
>>>>>>>>             1. Physical port
>>>>>>>>             2. Port channel
>>>>>>>>             3. Vlan.
>>>>>>>>
>>>>>>>>
>>>>>>>> Could we relax the uniqueness requirements of observationPointId
>>>>>>>> when
>>>>>>>> an observationPointType is also exported, such that the {
>>>>>>>> observationPointType, observationPointId } pair must be unique,
>>>>>>>> though
>>>>>>>> not the observationPointId itself.
>>>>>>>>
>>>>>>>> The reason being that observation points of different types
>>>>>>>>already
>>>>>>>> have IDs, though these are not unique. eg, we have interface 1,
>>>>>>>> 2, 3,
>>>>>>>> ...; vlan 1, 2, 3, ...; port 1, 2, 3, ...
>>>>>>>>
>>>>>>>> So an extra mediation layer is required to convert values from
>>>>>>>> each of
>>>>>>>> these number spaces into unique IDs in the observationPointId
>>>>>>>> number
>>>>>>>> space - which becomes harder as we add more places that
>>>>>>>> observations
>>>>>>>> can be made, ie more observationPointTypes.
>>>>>>>>
>>>>>>>> Whereas prefixing with the observationPointType achieves the
>>>>>>>> required
>>>>>>>> uniqueness in a faster, simpler, and more reliable way.
>>>>>>>>
>>>>>>>> To achieve this, I'd propose the following definition change:
>>>>>>>>
>>>>>>>>            An identifier of an Observation Point that is unique
>>>>>>>>per
>>>>>>>>            Observation Domain and per observationPointType, if
>>>>>>>> specified.
>>>>>>>>            It is RECOMMENDED that this identifier is
>>>>>>>>            also unique per IPFIX Device.  Typically, this
>>>>>>>> Information
>>>>>>>>            Element is used for limiting the scope of other
>>>>>>>> Information
>>>>>>>>            Elements.
>>>>>>>>
>>>>>>>> For backwards compatibility, we could define
>>>>>>>> observationPointType = 0
>>>>>>>> to be "unspecified".
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> P.
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> IPFIX mailing list
>>>>>>>> [email protected]
>>>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> IPFIX mailing list
>>>>>>> [email protected]
>>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>>>
>>>>>
>>>
>
>_______________________________________________
>IPFIX mailing list
>[email protected]
>https://www.ietf.org/mailman/listinfo/ipfix


_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix