Re: Second WGLC on draft-ietf-bmwg-traffic-management

Hi Al,

Looking at another comment in the area of repeatability:

[ACM]
> The new procedures add considerable needed details, but also raise a 
> few questions.
> 
> When considering repeatability over multiple test runs, there could be 
> a statistical measure of deviation for the key benchmarks so that the 
> audience can easily assess the degree of consistency observed.

The test is divided into pure packet, stateless generation with tradition metrics such as loss, delay jitter and stateful TCP traffic with RFC 6349 metrics.  As we both are aware, TCP variation will be greater than traditional packet based.

I'm struggling a little how to phrase the measure of variation even for the traditional (Scott Bradnor please chime in).

Let me throw out, 1% deviation for stateless and 3% for stateful tests as guidance for acceptable repeatability?

Thank you,
Barry Constantine

JDSU Network and Service Enablement
Principal Member Technical Staff
301-325-7069


-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[email protected]] 
Sent: Thursday, January 15, 2015 7:58 AM
To: Barry Constantine; [email protected]
Subject: RE: Second WGLC on draft-ietf-bmwg-traffic-management

you're right, apparently I pasted this comment one too many times...

> -----Original Message-----
> From: Barry Constantine [mailto:[email protected]]
> Sent: Thursday, January 15, 2015 7:52 AM
> To: MORTON, ALFRED C (AL); [email protected]
> Subject: RE: Second WGLC on draft-ietf-bmwg-traffic-management
> 
> Hi Al,
> 
> Concerning the several comments in the area of Measurement units, I am 
> unsure what is intended for some of these.  One example below:
> 
>    - Shaper Rate (SR): the Shaper Rate is only applicable to the  
> traffic shaping tests.  The SR represents the average egress output 
> rate (bps) over the test interval.
> 
> ACM] determine and add:
> Measurement Units for reported values.
> 
> In this example, the SR is bps, so sorry I must be missing something 
> obvious concerning your request.
> 
> Thanks,
> Barry
> 
> ________________________________________
> From: bmwg [[email protected]] on behalf of MORTON, ALFRED C (AL) 
> [[email protected]]
> Sent: Monday, January 05, 2015 2:10 PM
> To: [email protected]; [email protected]
> Subject: Re: [bmwg] Second WGLC on draft-ietf-bmwg-traffic-management
> 
> BMWG -FYI this WGLC ends tomorrow.
> 
> Hi Barry and Ramki,
> 
> These are my comments on
>                       Traffic Management Benchmarking
>                       draft-ietf-bmwg-traffic-management-01.txt
> 
> Al
> (as participant)
> 
> Note: as Document Shepherd, I see quite a few nits generated by the 
> nits-checker:
> https://tools.ietf.org/idnits?url=https://tools.ietf.org/id/draft-ietf
> -
> bmwg-traffic-management-01.txt
> 
> 
> High-level:
>    7. Security Considerations
> 
>    8. IANA Considerations
> 
>    9. Conclusions
> 
> Security and IANA sections are Mandatory, and cannot be blank.
> 
> I saw a few "TBD" in section 6, questions for Ramki. . .
> 
> The new procedures add considerable needed details, but also raise a 
> few questions.
> 
> When considering repeatability over multiple test runs, there could be 
> a statistical measure of deviation for the key benchmarks so that the 
> audience can easily assess the degree of consistency observed.
> 
> Detailed comments on a partial version of the draft are attached.
> 
> ________________________________________
> From: bmwg [[email protected]] On Behalf Of MORTON, ALFRED C (AL)
> Sent: Wednesday, December 10, 2014 2:30 PM
> To: [email protected]; [email protected]
> Subject: [bmwg] Second WGLC on draft-ietf-bmwg-traffic-management
> 
> BMWG (and AQM):
> 
> A WG Last Call period for the Internet-Draft on Traffic Management 
> Benchmarking:
> 
>    http://tools.ietf.org/html/draft-ietf-bmwg-traffic-management
> 
> will be open from 10 December 2014 through 6 January, 2015.
> 
> The first WGLC (on -00) closed October 21 2014, with substantial 
> comments and the authors believe they are now addressed.
> 
> This draft is continuing the BMWG Last Call Process. See 
> http://www1.ietf.org/mail-archive/web/bmwg/current/msg00846.html
> 
> Please read and express your opinion on whether or not this 
> Internet-Draft should be forwarded to the Area Directors for 
> publication as an Informational RFC.  Send your comments to this list 
> or [email protected] and [email protected]
> 
> Al
> bmwg co-chair
> 
> 
> _______________________________________________
> bmwg mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/bmwg