Re: [IPFIX] IPFIX observationPointId uniqueness

Gerhard,

This question needs input from collector experts on this list.

P.


On 27/03/13 20:54, Gerhard Muenz wrote:
> Paul,
>
> I do not see any problem with 64bit OP ID.
>
> I think, we had a similar motivation for having 64bit selector 
> IDs/selection sequence IDs.
>
> Will it confuse existing collectors? (hopefully not)
>
> Regards,
> Gerhard
>
>
> On 26.03.2013 13:20, Paul Aitken wrote:
>> Gerhard,
>>
>> My problem is mapping multiple discrete 32-bit number spaces into a
>> single 32-bit Observation Point ID.
>>
>> Keeping track of the mapping could be impossibly expensive: in the worst
>> case, 2^32 IDs x { (8-bit source type) + (32-bit source ID) + (32-bit
>> OPID) } = 36G of memory :-(
>>
>> This suggests that the number space should be carved, so source 1 uses
>> OP IDs 0 - 999, source 2 uses OP IDs 1,000 - 1,999 etc. However I can't
>> do that because there are only 32-bits in the destination (OP ID) number
>> space. Each of the source number spaces is also 32-bits; I can't know
>> whether they'll be linearly allocated or sparsely populated (eg, by a
>> hashing algorithm), so I can't impose that certain bits (eg the topmost
>> bits) be used for the carving.
>>
>> A 64-bit OP ID would really help me.
>>
>> Thanks,
>> P.
>>
>>
>> On 13/03/13 21:42, Gerhard Muenz wrote:
>>>
>>> Paul,
>>>
>>> I always thought of OPID, ODID, MPID, EPID etc. as being abstract
>>> identifiers. If an implementation can map them directly to
>>> implementation-specific internal parameters, that's fine. If not, a
>>> mapping needs to be provided by the device. That's the same for other
>>> data models.
>>>
>>> For example, you would also need to assign ipfixObservationPointIndex
>>> for each OP if you implement IPFIX-MIB. You could use the same value
>>> for the OPID IE.
>>>
>>> If there is no other solution than map 32 bit directly into OPID, I
>>> would prefer changing OPID to unsigned64.
>>>
>>> Regards,
>>> Gerhard
>>>
>>>
>>> On 13.03.2013 11:26, Paul Aitken wrote:
>>>> Gerhard,
>>>>
>>>> We are monitoring traffic in multiple places - at the ingress and 
>>>> egress
>>>> interfaces, at processes, in ACLs, in policies.
>>>>
>>>> Each of these has a unique 32-bit ID (ie, there may be up to N x 
>>>> 32-bit
>>>> IDs).
>>>>
>>>> However, these IDs are in unique number spaces. eg, the interface ID
>>>> could happen to be the same as the process ID or the same as the 
>>>> policy
>>>> ID. So it's not possible to say what the ID represents without knowing
>>>> the type.
>>>>
>>>> You would meet this issue if you monitor interfaces and also want to
>>>> report PSAMP stats for each selection process. Somehow you need to
>>>> report distinct IDs for the interfaces versus the processes.
>>>>
>>>> Several possibilities come to mind:
>>>>
>>>> 1. use multiple different IEs, one per observation point type, with
>>>> multiple templates. This is not ideal, because we have to request new
>>>> IEs for each new observation point type, and store / export multiple
>>>> templates. ie, it adds a layer of unnecessary complexity rather than
>>>> rather than simplifying the export.
>>>>
>>>> 2. implement a conversion function, mapping the N x 32-bit IDs into 
>>>> the
>>>> 1 x 32-bit observationDomainID space. Obviously only a limited 
>>>> number of
>>>> conversions (1/N) are possible, and there'll be a trade-off between 
>>>> the
>>>> table size and lookup time.
>>>>
>>>> 3. associate an observationPointType with each observationPointID, and
>>>> not require any conversion at all. Each internal ID is mapped 1:1 with
>>>> an observationPointId, with uniqueness being determined by the
>>>> associated observationPointType. This is the simplest and cleanest
>>>> solution.
>>>>
>>>>
>>>> So to answer your questions directly:
>>>>
>>>> - Do you really need OPID for your purpose? Have you thought about 
>>>> using
>>>> IEs like ingressPhysicalInterface or ingressInterface? Or, if these 
>>>> IEs
>>>> are not appropriate, about defining new IEs?
>>>>
>>>>        - per (1) above, this makes for an unnecessarily complex 
>>>> solution.
>>>>
>>>>
>>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>>> you thought about assigning the different sources (interface cards,
>>>> VLANs) to different ODs?
>>>>
>>>>        - they are all within a single OD.
>>>>        - Example use case: report packet counts as traffic progresses
>>>> through selection and filtering within the router. The report will be
>>>> exported in a single message with a single OD in the header.
>>>>            It's not possible to export a single cohesive report using
>>>> multiple ODs.
>>>>
>>>> P.
>>>>
>>>>
>>>>
>>>> On 15/02/13 09:25, Gerhard Muenz wrote:
>>>>>
>>>>> Benoit, Paul,
>>>>>
>>>>> Like Benoit, I do not have a good feeling about this proposal. On the
>>>>> other hand, I have not found any RFC that really requires the use of
>>>>> OPIDs. Even RFC5476 leaves it open whether the OP is identified by
>>>>> OPID or any other IE. Therefore, I have been silent until now.
>>>>>
>>>>> IPFIX-CONFIG assumes that the OPID is assigned by the IPFIX device,
>>>>> i.e. it is not a configuration parameter. I think that we agree here.
>>>>>
>>>>> Up to now, the IPFIX RFCs do not impose any restriction on the OPID
>>>>> except for the uniqueness per OD. This means that various
>>>>> implementations are possible right now:
>>>>> 1) OPID is equal to an internal identifier (e.g. internal interface
>>>>> number) or a non-IPFIX MIB index (e.g. ifIndex, entPhysicalIndex).
>>>>> 2) OPID is equal to the IPFIX-MIB index ipfixObservationPointIndex
>>>>> 3) OPID is unrelated to any other index, the IPFIX device provides 
>>>>> the
>>>>> internal mapping
>>>>>
>>>>> If IPFIX-MIB is supported by the device, the natural choice for me
>>>>> would be 2). This would allow easy mapping of MIB entries and IPFIX
>>>>> export.
>>>>>
>>>>> As Paul points out, 1) only works as long as the internal identifiers
>>>>> are unique per OD.
>>>>>
>>>>> My questions to Paul are:
>>>>> - Do you really need OPID for your purpose? Have you thought about
>>>>> using IEs like ingressPhysicalInterface or ingressInterface? Or, if
>>>>> these IEs are not appropriate, about defining new IEs?
>>>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>>>> you thought about assigning the different sources (interface cards,
>>>>> VLANs) to different ODs?
>>>>>
>>>>> Regards,
>>>>> Gerhard
>>>>>
>>>>>
>>>>> On 15.02.2013 01:23, Benoit Claise wrote:
>>>>>> Paul,
>>>>>>
>>>>>> At first glance, it looks reasonable.
>>>>>> But what would be the consequence on the Selection Sequence Report
>>>>>> Interpretation(see
>>>>>> http://tools.ietf.org/html/rfc5476#section-6.5.1). If
>>>>>> we use the observationPointId and it's not unique, then we need to
>>>>>> include the observationPointType?
>>>>>> Note that the sentence "It is RECOMMENDED that this identifier is 
>>>>>> also
>>>>>> unique per IPFIX Device." was inserted specifically for this.
>>>>>>
>>>>>> Regards, Benot
>>>>>>> Dear IPFIX experts,
>>>>>>>
>>>>>>> IPFIX observationPointId (#138) is defined as:
>>>>>>>
>>>>>>>            An identifier of an Observation Point that is unique per
>>>>>>>            Observation Domain.  It is RECOMMENDED that this
>>>>>>> identifier is
>>>>>>>            also unique per IPFIX Device.  Typically, this 
>>>>>>> Information
>>>>>>>            Element is used for limiting the scope of other 
>>>>>>> Information
>>>>>>>            Elements.
>>>>>>>
>>>>>>> We now also have observationPointType (#277), defined as:
>>>>>>>
>>>>>>>             Type of observation point. Values assigned to date are:
>>>>>>>
>>>>>>>             1. Physical port
>>>>>>>             2. Port channel
>>>>>>>             3. Vlan.
>>>>>>>
>>>>>>>
>>>>>>> Could we relax the uniqueness requirements of observationPointId 
>>>>>>> when
>>>>>>> an observationPointType is also exported, such that the {
>>>>>>> observationPointType, observationPointId } pair must be unique,
>>>>>>> though
>>>>>>> not the observationPointId itself.
>>>>>>>
>>>>>>> The reason being that observation points of different types already
>>>>>>> have IDs, though these are not unique. eg, we have interface 1, 
>>>>>>> 2, 3,
>>>>>>> ...; vlan 1, 2, 3, ...; port 1, 2, 3, ...
>>>>>>>
>>>>>>> So an extra mediation layer is required to convert values from
>>>>>>> each of
>>>>>>> these number spaces into unique IDs in the observationPointId 
>>>>>>> number
>>>>>>> space - which becomes harder as we add more places that 
>>>>>>> observations
>>>>>>> can be made, ie more observationPointTypes.
>>>>>>>
>>>>>>> Whereas prefixing with the observationPointType achieves the 
>>>>>>> required
>>>>>>> uniqueness in a faster, simpler, and more reliable way.
>>>>>>>
>>>>>>> To achieve this, I'd propose the following definition change:
>>>>>>>
>>>>>>>            An identifier of an Observation Point that is unique per
>>>>>>>            Observation Domain and per observationPointType, if
>>>>>>> specified.
>>>>>>>            It is RECOMMENDED that this identifier is
>>>>>>>            also unique per IPFIX Device.  Typically, this 
>>>>>>> Information
>>>>>>>            Element is used for limiting the scope of other 
>>>>>>> Information
>>>>>>>            Elements.
>>>>>>>
>>>>>>> For backwards compatibility, we could define 
>>>>>>> observationPointType = 0
>>>>>>> to be "unspecified".
>>>>>>>
>>>>>>> Thanks,
>>>>>>> P.
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> IPFIX mailing list
>>>>>>> [email protected]
>>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPFIX mailing list
>>>>>> [email protected]
>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>>
>>>>
>>

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix