FW: Last Call: 'Real-time Application Quality of Servic e Monitoring (RAQMON) Framework' to Proposed Standard (and other RAQMON d rafts)

forwardewd with permission.
Pls copy [email protected] when responding.

Alan, I am in the middle of a meeting, so possibly the WG chair
and/or authors will repsond before I do.

Bert

-----Original Message-----
From: [email protected] [mailto:[email protected]]On Behalf Of
Alan Clark
Sent: Sunday, February 12, 2006 22:29
To: [email protected]
Subject: Re: [RMONMIB] Last Call: 'Real-time Application Quality of
Service Monitoring (RAQMON) Framework' to Proposed Standard (and other
RAQMON drafts)



I have been an active reviewer of the RAQMON drafts during their 
progression through the RMONMIB WG, and have made numerous technical 
comments, most of which have been incorporated.  My concerns are firstly 
that there are some fundamental technical flaws with certain key 
metrics, secondly that some metric definitions are not defined in a 
manner consistent with other IETF standards.  I have pointed these out 
on numerous occasions however there has been reluctance to incorporate 
reasonable technical comment - although in fairness to the authors they 
did incorporate many of my other comments.

One simple but very fundamental technical issue relates to the 
definition of packets received and/or lost, which is one of the most 
basic and fundamental QoS metrics.  Unlike other standards that define 
packet counts - RAQMON uses the same counter for both signaling and 
media packets

 From the RAQMON Framework draft:

5.16 Total Number of Application Packets Received
This metric reports the number of application payload packets received 
by the RDS as part of this session since the last RAQMON PDU was sent up 
until the time this RAQMON PDU was generated. This parameter represents 
a very simple incremental counter that counts the number of 
"application" packets that an RDS has received. Applications packets MAY 
include signaling packets. Since this count is a snapshot in time, 
depending on application type, it also varies based on the application 
states e.g. an RDS within an application session will report aggregated 
number of application packets that were sent out during signaling setup, 
media packets received, session termination etc.
--------------------------

There are other similar counters for packets lost and cumulative packets 
received and lost.  These counters are also incorporated into 
Min/Avg/Max metrics reported through the RAQMON MIB.  

My concern with this is simple - the most fundamental set of metrics 
that are needed for QoS reporting - packet received and loss counts - is 
a kludgy hybrid of counts of signaling packets sent probably over TCP 
and media packets sent using RTP/UDP (which may also have taken 
different routes through the network). 
- The counter "may" or may not include signaling
- It is not clear which counters get reset after the signaling phase
- The counters can only be interpreted if you know the state the 
endpoint was in at the point the count was reported
- It is entirely unclear how to interpret these metrics once they are 
incorporated into Min/Max/Avg values in the MIB.

I pointed this out to the authors almost two years ago - and have 
repeated the comment several times:

Posted to RMONMIB WG list on 10th July 2004

5.14 Application Packets Received
Personally I still find this metric very confusing.  During a call the 
count would initially be signaling packets and then would switch to 
media packets:
- does the count get reset?
- does the count need to be interpreted in conjunction with the call 
state parameter?
- does this mean that a RAQMON report has to be sent whenever the call 
state changes?
- if the count includes both signaling and media packets then you can't 
refer to RFC3550 as describing how to count them as this only relates to 
RTP PDUs

I have pointed out to the authors that a simple way to address this 
would have been to use separate counters for signaling and media path.  
This would be very simple to do, can be interpreted independently of the 
state of the endpoint, is still intelligible after being 
aggregated........  My basic concern has remained unaddressed since July 
2004 and I can conceive of no good technical justification for keeping 
these definitions as they are.

Regards

Alan Clark



The IESG wrote:

>The IESG has received a request from the Remote Network Monitoring WG to 
>consider the following documents:
>
>- 'Real-time Application Quality of Service Monitoring (RAQMON) MIB '
>   <draft-ietf-rmonmib-raqmon-mib-11.txt> as a Proposed Standard
>- 'Real-time Application Quality of Service Monitoring (RAQMON) Framework '
>   <draft-ietf-rmonmib-raqmon-framework-14.txt> as a Proposed Standard
>- 'Transport Mappings for Real-time Application Quality of Service Monitoring 
>   (RAQMON) Protocol Data Unit (PDU) '
>   <draft-ietf-rmonmib-raqmon-pdu-12.txt> as a Proposed Standard
>
>   There is a potential overlap between the RAQMON protocol and
>   the IPFIX and/or RTCP-XR work, but there are enough differences
>   in applicability and complexity that this should not be
>   a significant issue.  RAQMON is an extensible, distributed,
>   SNMP-based monitoring system, integrated with the RMON
>   family of MIB modules.  As such, it is intended to be applicable
>   to many different types of protocols, that may run concurrently.  
>   If a specialized protocol for VoIP monitoring is desired, then 
>   RTCP-XR should probably be used instead.  If SNMP and/or 
>   RMON integration are not desired, then IPFIX would be a
>   suitable choice.
>
>The IESG plans to make a decision in the next few weeks, and solicits
>final comments on this action.  Please send any comments to the
>[email protected] or [email protected] mailing lists by 2006-02-13.
>
>The files can be obtained via
>http://www.ietf.org/internet-drafts/draft-ietf-rmonmib-raqmon-mib-11.txt
>http://www.ietf.org/internet-drafts/draft-ietf-rmonmib-raqmon-framework-14.txt
>http://www.ietf.org/internet-drafts/draft-ietf-rmonmib-raqmon-pdu-12.txt
>
>
>_______________________________________________
>RMONMIB mailing list
>[email protected]
>https://www1.ietf.org/mailman/listinfo/rmonmib
>
>  
>