Re: [IPFIX] IPFIX observationPointId uniqueness

Paul,

I do not see any problem with 64bit OP ID.

I think, we had a similar motivation for having 64bit selector 
IDs/selection sequence IDs.

Will it confuse existing collectors? (hopefully not)

Regards,
Gerhard


On 26.03.2013 13:20, Paul Aitken wrote:
> Gerhard,
>
> My problem is mapping multiple discrete 32-bit number spaces into a
> single 32-bit Observation Point ID.
>
> Keeping track of the mapping could be impossibly expensive: in the worst
> case, 2^32 IDs x { (8-bit source type) + (32-bit source ID) + (32-bit
> OPID) } = 36G of memory :-(
>
> This suggests that the number space should be carved, so source 1 uses
> OP IDs 0 - 999, source 2 uses OP IDs 1,000 - 1,999 etc. However I can't
> do that because there are only 32-bits in the destination (OP ID) number
> space. Each of the source number spaces is also 32-bits; I can't know
> whether they'll be linearly allocated or sparsely populated (eg, by a
> hashing algorithm), so I can't impose that certain bits (eg the topmost
> bits) be used for the carving.
>
> A 64-bit OP ID would really help me.
>
> Thanks,
> P.
>
>
> On 13/03/13 21:42, Gerhard Muenz wrote:
>>
>> Paul,
>>
>> I always thought of OPID, ODID, MPID, EPID etc. as being abstract
>> identifiers. If an implementation can map them directly to
>> implementation-specific internal parameters, that's fine. If not, a
>> mapping needs to be provided by the device. That's the same for other
>> data models.
>>
>> For example, you would also need to assign ipfixObservationPointIndex
>> for each OP if you implement IPFIX-MIB. You could use the same value
>> for the OPID IE.
>>
>> If there is no other solution than map 32 bit directly into OPID, I
>> would prefer changing OPID to unsigned64.
>>
>> Regards,
>> Gerhard
>>
>>
>> On 13.03.2013 11:26, Paul Aitken wrote:
>>> Gerhard,
>>>
>>> We are monitoring traffic in multiple places - at the ingress and egress
>>> interfaces, at processes, in ACLs, in policies.
>>>
>>> Each of these has a unique 32-bit ID (ie, there may be up to N x 32-bit
>>> IDs).
>>>
>>> However, these IDs are in unique number spaces. eg, the interface ID
>>> could happen to be the same as the process ID or the same as the policy
>>> ID. So it's not possible to say what the ID represents without knowing
>>> the type.
>>>
>>> You would meet this issue if you monitor interfaces and also want to
>>> report PSAMP stats for each selection process. Somehow you need to
>>> report distinct IDs for the interfaces versus the processes.
>>>
>>> Several possibilities come to mind:
>>>
>>> 1. use multiple different IEs, one per observation point type, with
>>> multiple templates. This is not ideal, because we have to request new
>>> IEs for each new observation point type, and store / export multiple
>>> templates. ie, it adds a layer of unnecessary complexity rather than
>>> rather than simplifying the export.
>>>
>>> 2. implement a conversion function, mapping the N x 32-bit IDs into the
>>> 1 x 32-bit observationDomainID space. Obviously only a limited number of
>>> conversions (1/N) are possible, and there'll be a trade-off between the
>>> table size and lookup time.
>>>
>>> 3. associate an observationPointType with each observationPointID, and
>>> not require any conversion at all. Each internal ID is mapped 1:1 with
>>> an observationPointId, with uniqueness being determined by the
>>> associated observationPointType. This is the simplest and cleanest
>>> solution.
>>>
>>>
>>> So to answer your questions directly:
>>>
>>> - Do you really need OPID for your purpose? Have you thought about using
>>> IEs like ingressPhysicalInterface or ingressInterface? Or, if these IEs
>>> are not appropriate, about defining new IEs?
>>>
>>>        - per (1) above, this makes for an unnecessarily complex solution.
>>>
>>>
>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>> you thought about assigning the different sources (interface cards,
>>> VLANs) to different ODs?
>>>
>>>        - they are all within a single OD.
>>>        - Example use case: report packet counts as traffic progresses
>>> through selection and filtering within the router. The report will be
>>> exported in a single message with a single OD in the header.
>>>            It's not possible to export a single cohesive report using
>>> multiple ODs.
>>>
>>> P.
>>>
>>>
>>>
>>> On 15/02/13 09:25, Gerhard Muenz wrote:
>>>>
>>>> Benoit, Paul,
>>>>
>>>> Like Benoit, I do not have a good feeling about this proposal. On the
>>>> other hand, I have not found any RFC that really requires the use of
>>>> OPIDs. Even RFC5476 leaves it open whether the OP is identified by
>>>> OPID or any other IE. Therefore, I have been silent until now.
>>>>
>>>> IPFIX-CONFIG assumes that the OPID is assigned by the IPFIX device,
>>>> i.e. it is not a configuration parameter. I think that we agree here.
>>>>
>>>> Up to now, the IPFIX RFCs do not impose any restriction on the OPID
>>>> except for the uniqueness per OD. This means that various
>>>> implementations are possible right now:
>>>> 1) OPID is equal to an internal identifier (e.g. internal interface
>>>> number) or a non-IPFIX MIB index (e.g. ifIndex, entPhysicalIndex).
>>>> 2) OPID is equal to the IPFIX-MIB index ipfixObservationPointIndex
>>>> 3) OPID is unrelated to any other index, the IPFIX device provides the
>>>> internal mapping
>>>>
>>>> If IPFIX-MIB is supported by the device, the natural choice for me
>>>> would be 2). This would allow easy mapping of MIB entries and IPFIX
>>>> export.
>>>>
>>>> As Paul points out, 1) only works as long as the internal identifiers
>>>> are unique per OD.
>>>>
>>>> My questions to Paul are:
>>>> - Do you really need OPID for your purpose? Have you thought about
>>>> using IEs like ingressPhysicalInterface or ingressInterface? Or, if
>>>> these IEs are not appropriate, about defining new IEs?
>>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>>> you thought about assigning the different sources (interface cards,
>>>> VLANs) to different ODs?
>>>>
>>>> Regards,
>>>> Gerhard
>>>>
>>>>
>>>> On 15.02.2013 01:23, Benoit Claise wrote:
>>>>> Paul,
>>>>>
>>>>> At first glance, it looks reasonable.
>>>>> But what would be the consequence on the Selection Sequence Report
>>>>> Interpretation(see
>>>>> http://tools.ietf.org/html/rfc5476#section-6.5.1). If
>>>>> we use the observationPointId and it's not unique, then we need to
>>>>> include the observationPointType?
>>>>> Note that the sentence "It is RECOMMENDED that this identifier is also
>>>>> unique per IPFIX Device." was inserted specifically for this.
>>>>>
>>>>> Regards, Benot
>>>>>> Dear IPFIX experts,
>>>>>>
>>>>>> IPFIX observationPointId (#138) is defined as:
>>>>>>
>>>>>>            An identifier of an Observation Point that is unique per
>>>>>>            Observation Domain.  It is RECOMMENDED that this
>>>>>> identifier is
>>>>>>            also unique per IPFIX Device.  Typically, this Information
>>>>>>            Element is used for limiting the scope of other Information
>>>>>>            Elements.
>>>>>>
>>>>>> We now also have observationPointType (#277), defined as:
>>>>>>
>>>>>>             Type of observation point. Values assigned to date are:
>>>>>>
>>>>>>             1. Physical port
>>>>>>             2. Port channel
>>>>>>             3. Vlan.
>>>>>>
>>>>>>
>>>>>> Could we relax the uniqueness requirements of observationPointId when
>>>>>> an observationPointType is also exported, such that the {
>>>>>> observationPointType, observationPointId } pair must be unique,
>>>>>> though
>>>>>> not the observationPointId itself.
>>>>>>
>>>>>> The reason being that observation points of different types already
>>>>>> have IDs, though these are not unique. eg, we have interface 1, 2, 3,
>>>>>> ...; vlan 1, 2, 3, ...; port 1, 2, 3, ...
>>>>>>
>>>>>> So an extra mediation layer is required to convert values from
>>>>>> each of
>>>>>> these number spaces into unique IDs in the observationPointId number
>>>>>> space - which becomes harder as we add more places that observations
>>>>>> can be made, ie more observationPointTypes.
>>>>>>
>>>>>> Whereas prefixing with the observationPointType achieves the required
>>>>>> uniqueness in a faster, simpler, and more reliable way.
>>>>>>
>>>>>> To achieve this, I'd propose the following definition change:
>>>>>>
>>>>>>            An identifier of an Observation Point that is unique per
>>>>>>            Observation Domain and per observationPointType, if
>>>>>> specified.
>>>>>>            It is RECOMMENDED that this identifier is
>>>>>>            also unique per IPFIX Device.  Typically, this Information
>>>>>>            Element is used for limiting the scope of other Information
>>>>>>            Elements.
>>>>>>
>>>>>> For backwards compatibility, we could define observationPointType = 0
>>>>>> to be "unspecified".
>>>>>>
>>>>>> Thanks,
>>>>>> P.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPFIX mailing list
>>>>>> [email protected]
>>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPFIX mailing list
>>>>> [email protected]
>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>>
>>>
>
_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix