Re: [IPFIX] IPFIX observationPointId uniqueness

Gerhard,

My problem is mapping multiple discrete 32-bit number spaces into a 
single 32-bit Observation Point ID.

Keeping track of the mapping could be impossibly expensive: in the worst 
case, 2^32 IDs x { (8-bit source type) + (32-bit source ID) + (32-bit 
OPID) } = 36G of memory :-(

This suggests that the number space should be carved, so source 1 uses 
OP IDs 0 - 999, source 2 uses OP IDs 1,000 - 1,999 etc. However I can't 
do that because there are only 32-bits in the destination (OP ID) number 
space. Each of the source number spaces is also 32-bits; I can't know 
whether they'll be linearly allocated or sparsely populated (eg, by a 
hashing algorithm), so I can't impose that certain bits (eg the topmost 
bits) be used for the carving.

A 64-bit OP ID would really help me.

Thanks,
P.


On 13/03/13 21:42, Gerhard Muenz wrote:
>
> Paul,
>
> I always thought of OPID, ODID, MPID, EPID etc. as being abstract 
> identifiers. If an implementation can map them directly to 
> implementation-specific internal parameters, that's fine. If not, a 
> mapping needs to be provided by the device. That's the same for other 
> data models.
>
> For example, you would also need to assign ipfixObservationPointIndex 
> for each OP if you implement IPFIX-MIB. You could use the same value 
> for the OPID IE.
>
> If there is no other solution than map 32 bit directly into OPID, I 
> would prefer changing OPID to unsigned64.
>
> Regards,
> Gerhard
>
>
> On 13.03.2013 11:26, Paul Aitken wrote:
>> Gerhard,
>>
>> We are monitoring traffic in multiple places - at the ingress and egress
>> interfaces, at processes, in ACLs, in policies.
>>
>> Each of these has a unique 32-bit ID (ie, there may be up to N x 32-bit
>> IDs).
>>
>> However, these IDs are in unique number spaces. eg, the interface ID
>> could happen to be the same as the process ID or the same as the policy
>> ID. So it's not possible to say what the ID represents without knowing
>> the type.
>>
>> You would meet this issue if you monitor interfaces and also want to
>> report PSAMP stats for each selection process. Somehow you need to
>> report distinct IDs for the interfaces versus the processes.
>>
>> Several possibilities come to mind:
>>
>> 1. use multiple different IEs, one per observation point type, with
>> multiple templates. This is not ideal, because we have to request new
>> IEs for each new observation point type, and store / export multiple
>> templates. ie, it adds a layer of unnecessary complexity rather than
>> rather than simplifying the export.
>>
>> 2. implement a conversion function, mapping the N x 32-bit IDs into the
>> 1 x 32-bit observationDomainID space. Obviously only a limited number of
>> conversions (1/N) are possible, and there'll be a trade-off between the
>> table size and lookup time.
>>
>> 3. associate an observationPointType with each observationPointID, and
>> not require any conversion at all. Each internal ID is mapped 1:1 with
>> an observationPointId, with uniqueness being determined by the
>> associated observationPointType. This is the simplest and cleanest 
>> solution.
>>
>>
>> So to answer your questions directly:
>>
>> - Do you really need OPID for your purpose? Have you thought about using
>> IEs like ingressPhysicalInterface or ingressInterface? Or, if these IEs
>> are not appropriate, about defining new IEs?
>>
>>       - per (1) above, this makes for an unnecessarily complex solution.
>>
>>
>> - If you need to use OPIDs and if you want to avoid any mapping, have
>> you thought about assigning the different sources (interface cards,
>> VLANs) to different ODs?
>>
>>       - they are all within a single OD.
>>       - Example use case: report packet counts as traffic progresses
>> through selection and filtering within the router. The report will be
>> exported in a single message with a single OD in the header.
>>           It's not possible to export a single cohesive report using
>> multiple ODs.
>>
>> P.
>>
>>
>>
>> On 15/02/13 09:25, Gerhard Muenz wrote:
>>>
>>> Benoit, Paul,
>>>
>>> Like Benoit, I do not have a good feeling about this proposal. On the
>>> other hand, I have not found any RFC that really requires the use of
>>> OPIDs. Even RFC5476 leaves it open whether the OP is identified by
>>> OPID or any other IE. Therefore, I have been silent until now.
>>>
>>> IPFIX-CONFIG assumes that the OPID is assigned by the IPFIX device,
>>> i.e. it is not a configuration parameter. I think that we agree here.
>>>
>>> Up to now, the IPFIX RFCs do not impose any restriction on the OPID
>>> except for the uniqueness per OD. This means that various
>>> implementations are possible right now:
>>> 1) OPID is equal to an internal identifier (e.g. internal interface
>>> number) or a non-IPFIX MIB index (e.g. ifIndex, entPhysicalIndex).
>>> 2) OPID is equal to the IPFIX-MIB index ipfixObservationPointIndex
>>> 3) OPID is unrelated to any other index, the IPFIX device provides the
>>> internal mapping
>>>
>>> If IPFIX-MIB is supported by the device, the natural choice for me
>>> would be 2). This would allow easy mapping of MIB entries and IPFIX
>>> export.
>>>
>>> As Paul points out, 1) only works as long as the internal identifiers
>>> are unique per OD.
>>>
>>> My questions to Paul are:
>>> - Do you really need OPID for your purpose? Have you thought about
>>> using IEs like ingressPhysicalInterface or ingressInterface? Or, if
>>> these IEs are not appropriate, about defining new IEs?
>>> - If you need to use OPIDs and if you want to avoid any mapping, have
>>> you thought about assigning the different sources (interface cards,
>>> VLANs) to different ODs?
>>>
>>> Regards,
>>> Gerhard
>>>
>>>
>>> On 15.02.2013 01:23, Benoit Claise wrote:
>>>> Paul,
>>>>
>>>> At first glance, it looks reasonable.
>>>> But what would be the consequence on the Selection Sequence Report
>>>> Interpretation(see 
>>>> http://tools.ietf.org/html/rfc5476#section-6.5.1). If
>>>> we use the observationPointId and it's not unique, then we need to
>>>> include the observationPointType?
>>>> Note that the sentence "It is RECOMMENDED that this identifier is also
>>>> unique per IPFIX Device." was inserted specifically for this.
>>>>
>>>> Regards, Benot
>>>>> Dear IPFIX experts,
>>>>>
>>>>> IPFIX observationPointId (#138) is defined as:
>>>>>
>>>>>           An identifier of an Observation Point that is unique per
>>>>>           Observation Domain.  It is RECOMMENDED that this 
>>>>> identifier is
>>>>>           also unique per IPFIX Device.  Typically, this Information
>>>>>           Element is used for limiting the scope of other Information
>>>>>           Elements.
>>>>>
>>>>> We now also have observationPointType (#277), defined as:
>>>>>
>>>>>            Type of observation point. Values assigned to date are:
>>>>>
>>>>>            1. Physical port
>>>>>            2. Port channel
>>>>>            3. Vlan.
>>>>>
>>>>>
>>>>> Could we relax the uniqueness requirements of observationPointId when
>>>>> an observationPointType is also exported, such that the {
>>>>> observationPointType, observationPointId } pair must be unique, 
>>>>> though
>>>>> not the observationPointId itself.
>>>>>
>>>>> The reason being that observation points of different types already
>>>>> have IDs, though these are not unique. eg, we have interface 1, 2, 3,
>>>>> ...; vlan 1, 2, 3, ...; port 1, 2, 3, ...
>>>>>
>>>>> So an extra mediation layer is required to convert values from 
>>>>> each of
>>>>> these number spaces into unique IDs in the observationPointId number
>>>>> space - which becomes harder as we add more places that observations
>>>>> can be made, ie more observationPointTypes.
>>>>>
>>>>> Whereas prefixing with the observationPointType achieves the required
>>>>> uniqueness in a faster, simpler, and more reliable way.
>>>>>
>>>>> To achieve this, I'd propose the following definition change:
>>>>>
>>>>>           An identifier of an Observation Point that is unique per
>>>>>           Observation Domain and per observationPointType, if 
>>>>> specified.
>>>>>           It is RECOMMENDED that this identifier is
>>>>>           also unique per IPFIX Device.  Typically, this Information
>>>>>           Element is used for limiting the scope of other Information
>>>>>           Elements.
>>>>>
>>>>> For backwards compatibility, we could define observationPointType = 0
>>>>> to be "unspecified".
>>>>>
>>>>> Thanks,
>>>>> P.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPFIX mailing list
>>>>> [email protected]
>>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> IPFIX mailing list
>>>> [email protected]
>>>> https://www.ietf.org/mailman/listinfo/ipfix
>>>>
>>

_______________________________________________
IPFIX mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/ipfix