Re: [IPFIX] IPFIX observationPointId uniqueness

Thanks Andrew and Gerhard.

I'll forward the request to IE-doctors and IANA.

P.


On 28/03/13 12:01, Andrew Feren wrote:
> Hi Paul, Gerhard,
>
> I can't speak for other collectors, but I don't see a problem with
> increasing the length of observationPointId.
>
> I never saw any upside to doing anything other than respecting the length
> in a template when that length in the template differs from the nominal
> type size of the IE.  For reduced size encoding this is a specified
> behavior disallowing increased size encoding seemed like extra work with
> no return.
>
> Also I think Gerhard made a good case for increasing the size being a
> better option than the original proposal.  I didn't have an problem with
> the original proposal, but after considering Gerhard's arguments I think
> increasing the length of observationPointId is less likely to have any
> negative consequences.
>
> -Andrew
>
>
> On 3/28/13 5:52 AM, "Paul Aitken" <[email protected]> wrote:
>
>> Gerhard,
>>
>> This question needs input from collector experts on this list.
>>
>> P.
>>
>>
>> On 27/03/13 20:54, Gerhard Muenz wrote:
>>> Paul,
>>>
>>> I do not see any problem with 64bit OP ID.
>>>
>>> I think, we had a similar motivation for having 64bit selector
>>> IDs/selection sequence IDs.
>>>
>>> Will it confuse existing collectors? (hopefully not)
>>>
>>> Regards,
>>> Gerhard
>>>
>>>
>>> On 26.03.2013 13:20, Paul Aitken wrote:
>>>> Gerhard,
>>>>
>>>> My problem is mapping multiple discrete 32-bit number spaces into a
>>>> single 32-bit Observation Point ID.
>>>>
>>>> Keeping track of the mapping could be impossibly expensive: in the
>>>> worst
>>>> case, 2^32 IDs x { (8-bit source type) + (32-bit source ID) + (32-bit
>>>> OPID) } = 36G of memory :-(
>>>>
>>>> This suggests that the number space should be carved, so source 1 uses
>>>> OP IDs 0 - 999, source 2 uses OP IDs 1,000 - 1,999 etc. However I can't
>>>> do that because there are only 32-bits in the destination (OP ID)
>>>> number
>>>> space. Each of the source number spaces is also 32-bits; I can't know
>>>> whether they'll be linearly allocated or sparsely populated (eg, by a
>>>> hashing algorithm), so I can't impose that certain bits (eg the topmost
>>>> bits) be used for the carving.
>>>>
>>>> A 64-bit OP ID would really help me.
>>>>
>>>> Thanks,
>>>> P.
>>>>
>>>>
>>>> On 13/03/13 21:42, Gerhard Muenz wrote:
>>>>> Paul,
>>>>>
>>>>> I always thought of OPID, ODID, MPID, EPID etc. as being abstract
>>>>> identifiers. If an implementation can map them directly to
>>>>> implementation-specific internal parameters, that's fine. If not, a
>>>>> mapping needs to be provided by the device. That's the same for other
>>>>> data models.
>>>>>
>>>>> For example, you would also need to assign ipfixObservationPointIndex
>>>>> for each OP if you implement IPFIX-MIB. You could use the same value
>>>>> for the OPID IE.
>>>>>
>>>>> If there is no other solution than map 32 bit directly into OPID, I
>>>>> would prefer changing OPID to unsigned64.
>>>>>
>>>>> Regards,
>>>>> Gerhard
>>>>>
>>>>>
>>>>> On 13.03.2013 11:26, Paul Aitken wrote:
>>>>>> Gerhard,
>>>>>>
>>>>>> We are monitoring traffic in multiple places - at the ingress and
>>>>>> egress
>>>>>> interfaces, at processes, in ACLs, in policies.
>>>>>>
>>>>>> Each of these has a unique 32-bit ID (ie, there may be up to N x
>>>>>> 32-bit
>>>>>> IDs).
>>>>>>
>>>>>> However, these IDs are in unique number spaces. eg, the interface ID
>>>>>> could happen to be the same as the process ID or the same as the
>>>>>> policy
>>>>>> ID. So it's not possible to say what the ID represents without
>>>>>> knowing
>>>>>> the type.
>>>>>>
>>>>>> You would meet this issue if you monitor interfaces and also want to
>>>>>> report PSAMP stats for each selection process. Somehow you need to
>>>>>> report distinct IDs for the interfaces versus the processes.
>>>>>>
>>>>>> Several possibilities come to mind:
>>>>>>
>>>>>> 1. use multiple different IEs, one per observation point type, with
>>>>>> multiple templates. This is not ideal, because we have to request new
>>>>>> IEs for each new observation point type, and store / export multiple
>>>>>> templates. ie, it adds a layer of unnecessary complexity rather than
>>>>>> rather than simplifying the export.
>>>>>>
>>>>>> 2. implement a conversion function, mapping the N x 32-bit IDs into
>>>>>> the
>>>>>> 1 x 32-bit observationDomainID space. Obviously only a limited
>>>>>> number of
>>>>>> conversions (1/N) are possible, and there'll be a trade-off between
>>>>>> the
>>>>>> table size and lookup time.
>>>>>>
>>>>>> 3. associate an observationPointType with each observationPointID,
>>>>>> and
>>>>>> not require any conversion at all. Each internal ID is mapped 1:1
>>>>>> with
>>>>>> an observationPointId, with uniqueness being determined by the
>>>>>> associated observationPointType. This is the simplest and cleanest
>>>>>> solution.
>>>>>>
>>>>>>
>>>>>> So to answer your questions directly:
>>>>>>
>>>>>> - Do you really need OPID for your purpose? Have you thought about
>>>>>> using
>>>>>> IEs like ingressPhysicalInterface or ingressInterface? Or, if these
>>>>>> IEs
>>>>>> are not appropriate, about defining new IEs?
>>>>>>
>>>>>>         - per (1) above, this makes for an unnecessarily complex
>>>>>> solution.
>>>>>>
>>>>>>
>>>>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>>>>> you thought about assigning the different sources (interface cards,
>>>>>> VLANs) to different ODs?
>>>>>>
>>>>>>         - they are all within a single OD.
>>>>>>         - Example use case: report packet counts as traffic progresses
>>>>>> through selection and filtering within the router. The report will be
>>>>>> exported in a single message with a single OD in the header.
>>>>>>             It's not possible to export a single cohesive report using
>>>>>> multiple ODs.
>>>>>>
>>>>>> P.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 15/02/13 09:25, Gerhard Muenz wrote:
>>>>>>> Benoit, Paul,
>>>>>>>
>>>>>>> Like Benoit, I do not have a good feeling about this proposal. On
>>>>>>> the
>>>>>>> other hand, I have not found any RFC that really requires the use of
>>>>>>> OPIDs. Even RFC5476 leaves it open whether the OP is identified by
>>>>>>> OPID or any other IE. Therefore, I have been silent until now.
>>>>>>>
>>>>>>> IPFIX-CONFIG assumes that the OPID is assigned by the IPFIX device,
>>>>>>> i.e. it is not a configuration parameter. I think that we agree
>>>>>>> here.
>>>>>>>
>>>>>>> Up to now, the IPFIX RFCs do not impose any restriction on the OPID
>>>>>>> except for the uniqueness per OD. This means that various
>>>>>>> implementations are possible right now:
>>>>>>> 1) OPID is equal to an internal identifier (e.g. internal interface
>>>>>>> number) or a non-IPFIX MIB index (e.g. ifIndex, entPhysicalIndex).
>>>>>>> 2) OPID is equal to the IPFIX-MIB index ipfixObservationPointIndex
>>>>>>> 3) OPID is unrelated to any other index, the IPFIX device provides
>>>>>>> the
>>>>>>> internal mapping
>>>>>>>
>>>>>>> If IPFIX-MIB is supported by the device, the natural choice for me
>>>>>>> would be 2). This would allow easy mapping of MIB entries and IPFIX
>>>>>>> export.
>>>>>>>
>>>>>>> As Paul points out, 1) only works as long as the internal
>>>>>>> identifiers
>>>>>>> are unique per OD.
>>>>>>>
>>>>>>> My questions to Paul are:
>>>>>>> - Do you really need OPID for your purpose? Have you thought about
>>>>>>> using IEs like ingressPhysicalInterface or ingressInterface? Or, if
>>>>>>> these IEs are not appropriate, about defining new IEs?
>>>>>>> - If you need to use OPIDs and if you want to avoid any mapping,
>>>>>>> have
>>>>>>> you thought about assigning the different sources (interface cards,
>>>>>>> VLANs) to different ODs?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Gerhard
>>>>>>>
>>>>>>>
>>>>>>> On 15.02.2013 01:23, Benoit Claise wrote:
>>>>>>>> Paul,
>>>>>>>>
>>>>>>>> At first glance, it looks reasonable.
>>>>>>>> But what would be the consequence on the Selection Sequence Report
>>>>>>>> Interpretation(see
>>>>>>>> http://tools.ietf.org/html/rfc5476#section-6.5.1). If
>>>>>>>> we use the observationPointId and it's not unique, then we need to
>>>>>>>> include the observationPointType?
>>>>>>>> Note that the sentence "It is RECOMMENDED that this identifier is
>>>>>>>> also
>>>>>>>> unique per IPFIX Device." was inserted specifically for this.
>>>>>>>>
>>>>>>>> Regards, Benot
>>>>>>>>> Dear IPFIX experts,
>>>>>>>>>
>>>>>>>>> IPFIX observationPointId (#138) is defined as:
>>>>>>>>>
>>>>>>>>>             An identifier of an Observation Point that is unique
>>>>>>>>> per
>>>>>>>>>             Observation Domain.  It is RECOMMENDED that this
>>>>>>>>> identifier is
>>>>>>>>>             also unique per IPFIX Device.  Typically, this
>>>>>>>>> Information
>>>>>>>>>             Element is used for limiting the scope of other
>>>>>>>>> Information
>>>>>>>>>             Elements.
>>>>>>>>>
>>>>>>>>> We now also have observationPointType (#277), defined as:
>>>>>>>>>
>>>>>>>>>              Type of observation point. Values assigned to date
>>>>>>>>> are:
>>>>>>>>>
>>>>>>>>>              1. Physical port
>>>>>>>>>              2. Port channel
>>>>>>>>>              3. Vlan.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Could we relax the uniqueness requirements of observationPointId
>>>>>>>>> when
>>>>>>>>> an observationPointType is also exported, such that the {
>>>>>>>>> observationPointType, observationPointId } pair must be unique,
>>>>>>>>> though
>>>>>>>>> not the observationPointId itself.
>>>>>>>>>
>>>>>>>>> The reason being that observation points of different types
>>>>>>>>> already
>>>>>>>>> have IDs, though these are not unique. eg, we have interface 1,
>>>>>>>>> 2, 3,
>>>>>>>>> ...; vlan 1, 2, 3, ...; port 1, 2, 3, ...
>>>>>>>>>
>>>>>>>>> So an extra mediation layer is required to convert values from
>>>>>>>>> each of
>>>>>>>>> these number spaces into unique IDs in the observationPointId
>>>>>>>>> number
>>>>>>>>> space - which becomes harder as we add more places that
>>>>>>>>> observations
>>>>>>>>> can be made, ie more observationPointTypes.
>>>>>>>>>
>>>>>>>>> Whereas prefixing with the observationPointType achieves the
>>>>>>>>> required
>>>>>>>>> uniqueness in a faster, simpler, and more reliable way.
>>>>>>>>>
>>>>>>>>> To achieve this, I'd propose the following definition change:
>>>>>>>>>
>>>>>>>>>             An identifier of an Observation Point that is unique
>>>>>>>>> per
>>>>>>>>>             Observation Domain and per observationPointType, if
>>>>>>>>> specified.
>>>>>>>>>             It is RECOMMENDED that this identifier is
>>>>>>>>>             also unique per IPFIX Device.  Typically, this
>>>>>>>>> Information
>>>>>>>>>             Element is used for limiting the scope of other
>>>>>>>>> Information
>>>>>>>>>             Elements.
>>>>>>>>>
>>>>>>>>> For backwards compatibility, we could define
>>>>>>>>> observationPointType = 0
>>>>>>>>> to be "unspecified".
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> P.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> IPFIX mailing list
>>>>>>>>> [email protected]
>>>>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> IPFIX mailing list
>>>>>>>> [email protected]
>>>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>>>>
>> _______________________________________________
>> IPFIX mailing list
>> [email protected]
>> https://www.ietf.org/mailman/listinfo/ipfix

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix