Re: FW: Last Call: 'Real-time Application Quality of Servic e Monitoring (RAQMON) Framework' to Proposed Standard (and other RAQMON d rafts)

Wijnen, Bert (Bert) wrote:
> forwardewd with permission.
> Pls copy [email protected] when responding.
>
> Alan, I am in the middle of a meeting, so possibly the WG chair
> and/or authors will repsond before I do.
>   

This issue (of metrics) was discussed in the past by the Working Group,
and there was consensus that there is no need to separate application
packets received and transmitted into separate metrics for payload and
signaling.   This can be easily accomplished by defining additional
metrics for collection, beyond those found in the basic RAQMON report.


For many applications there is no separation of the application
traffic in payload and signaling. Even for those applications that
do carry distinct payload and signaling traffic most of the signaling
traffic happens at the initiation of the session. For these reasons
the decision was made to include only total application packets
(received and transmitted) in the basic standards metrics set.


It has never been a design goal of the RMONMIB WG to provide highly
detailed statistics for the monitoring of any specific protocol.
Instead, RMON provides a family of MIB modules that collect
related information about any protocol, and present that data
in a consistent manner via SNMP.  The fact that some additional
(more granular) statistics are interesting to collect for VoIP
really doesn't impact this overall architecture. 


Andy

> Bert
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]On Behalf Of
> Alan Clark
> Sent: Sunday, February 12, 2006 22:29
> To: [email protected]
> Subject: Re: [RMONMIB] Last Call: 'Real-time Application Quality of
> Service Monitoring (RAQMON) Framework' to Proposed Standard (and other
> RAQMON drafts)
>
>
>
> I have been an active reviewer of the RAQMON drafts during their 
> progression through the RMONMIB WG, and have made numerous technical 
> comments, most of which have been incorporated.  My concerns are firstly 
> that there are some fundamental technical flaws with certain key 
> metrics, secondly that some metric definitions are not defined in a 
> manner consistent with other IETF standards.  I have pointed these out 
> on numerous occasions however there has been reluctance to incorporate 
> reasonable technical comment - although in fairness to the authors they 
> did incorporate many of my other comments.
>
> One simple but very fundamental technical issue relates to the 
> definition of packets received and/or lost, which is one of the most 
> basic and fundamental QoS metrics.  Unlike other standards that define 
> packet counts - RAQMON uses the same counter for both signaling and 
> media packets
>
>  From the RAQMON Framework draft:
>
> 5.16 Total Number of Application Packets Received
> This metric reports the number of application payload packets received 
> by the RDS as part of this session since the last RAQMON PDU was sent up 
> until the time this RAQMON PDU was generated. This parameter represents 
> a very simple incremental counter that counts the number of 
> "application" packets that an RDS has received. Applications packets MAY 
> include signaling packets. Since this count is a snapshot in time, 
> depending on application type, it also varies based on the application 
> states e.g. an RDS within an application session will report aggregated 
> number of application packets that were sent out during signaling setup, 
> media packets received, session termination etc.
> --------------------------
>
> There are other similar counters for packets lost and cumulative packets 
> received and lost.  These counters are also incorporated into 
> Min/Avg/Max metrics reported through the RAQMON MIB.  
>
> My concern with this is simple - the most fundamental set of metrics 
> that are needed for QoS reporting - packet received and loss counts - is 
> a kludgy hybrid of counts of signaling packets sent probably over TCP 
> and media packets sent using RTP/UDP (which may also have taken 
> different routes through the network). 
> - The counter "may" or may not include signaling
> - It is not clear which counters get reset after the signaling phase
> - The counters can only be interpreted if you know the state the 
> endpoint was in at the point the count was reported
> - It is entirely unclear how to interpret these metrics once they are 
> incorporated into Min/Max/Avg values in the MIB.
>
> I pointed this out to the authors almost two years ago - and have 
> repeated the comment several times:
>
> Posted to RMONMIB WG list on 10th July 2004
>
> 5.14 Application Packets Received
> Personally I still find this metric very confusing.  During a call the 
> count would initially be signaling packets and then would switch to 
> media packets:
> - does the count get reset?
> - does the count need to be interpreted in conjunction with the call 
> state parameter?
> - does this mean that a RAQMON report has to be sent whenever the call 
> state changes?
> - if the count includes both signaling and media packets then you can't 
> refer to RFC3550 as describing how to count them as this only relates to 
> RTP PDUs
>
> I have pointed out to the authors that a simple way to address this 
> would have been to use separate counters for signaling and media path.  
> This would be very simple to do, can be interpreted independently of the 
> state of the endpoint, is still intelligible after being 
> aggregated........  My basic concern has remained unaddressed since July 
> 2004 and I can conceive of no good technical justification for keeping 
> these definitions as they are.
>
> Regards
>
> Alan Clark
>
>
>
> The IESG wrote:
>
>   
>> The IESG has received a request from the Remote Network Monitoring WG to 
>> consider the following documents:
>>
>> - 'Real-time Application Quality of Service Monitoring (RAQMON) MIB '
>>   <draft-ietf-rmonmib-raqmon-mib-11.txt> as a Proposed Standard
>> - 'Real-time Application Quality of Service Monitoring (RAQMON) Framework '
>>   <draft-ietf-rmonmib-raqmon-framework-14.txt> as a Proposed Standard
>> - 'Transport Mappings for Real-time Application Quality of Service Monitoring 
>>   (RAQMON) Protocol Data Unit (PDU) '
>>   <draft-ietf-rmonmib-raqmon-pdu-12.txt> as a Proposed Standard
>>
>>   There is a potential overlap between the RAQMON protocol and
>>   the IPFIX and/or RTCP-XR work, but there are enough differences
>>   in applicability and complexity that this should not be
>>   a significant issue.  RAQMON is an extensible, distributed,
>>   SNMP-based monitoring system, integrated with the RMON
>>   family of MIB modules.  As such, it is intended to be applicable
>>   to many different types of protocols, that may run concurrently.  
>>   If a specialized protocol for VoIP monitoring is desired, then 
>>   RTCP-XR should probably be used instead.  If SNMP and/or 
>>   RMON integration are not desired, then IPFIX would be a
>>   suitable choice.
>>
>> The IESG plans to make a decision in the next few weeks, and solicits
>> final comments on this action.  Please send any comments to the
>> [email protected] or [email protected] mailing lists by 2006-02-13.
>>
>> The files can be obtained via
>> http://www.ietf.org/internet-drafts/draft-ietf-rmonmib-raqmon-mib-11.txt
>> http://www.ietf.org/internet-drafts/draft-ietf-rmonmib-raqmon-framework-14.txt
>> http://www.ietf.org/internet-drafts/draft-ietf-rmonmib-raqmon-pdu-12.txt
>>
>>
>> _______________________________________________
>> RMONMIB mailing list
>> [email protected]
>> https://www1.ietf.org/mailman/listinfo/rmonmib
>>
>>  
>>
>>     
>
>
> _______________________________________________
> RMONMIB mailing list
> [email protected]
> https://www1.ietf.org/mailman/listinfo/rmonmib
>
>
>