Re: comments on draft-dcbench-def-00 (part 1)

Hi David,

Although RFC 3393 includes the absolute value, it's
only by way of reference to 1889 and 2598, and the
IPPM consensus was to deprecate the term jitter.
Thus the many other 3393 metrics for delay variation 
use differences without the absolute value, and 
they also supply some summary statistics in section 4.
I don't think they include the mean.

Also, in my example 
> <Data Center Engineers> wish to know the overall jitter from a path
> with ten DUTs in series, we can estimate that value but we have to
> measure the right metric at the start to do it.

I assumed we have benchmarked a single device, and wanted to
estimate how some number of these devices would affect delay 
variation when deployed on the path. Clearly, if we can benchmark
a particular black-box SUT with a specific combination of these devices,
there's no need to do the estimation for that case.
But having the ability to estimate e2e variation for an unknown number on the path
seems valuable to me, and I wonder if Data Center Engineers agree.

it's been a long day here, hope this makes sense,
Al



> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of
> Lucien Avramov (lavramov)
> Sent: Wednesday, July 31, 2013 4:27 AM
> To: David Newman
> Cc: [email protected]
> Subject: Re: [bmwg] comments on draft-dcbench-def-00 (part 1)
> 
> Hi David,
> 
> Please see inline
> 
> On 7/30/13 7:45 PM, David Newman wrote:
> > On 7/30/13 4:39 AM, MORTON JR., ALFRED C (AL) wrote:
> >
> >> the absolute value function used in 4689
> >> colapses the positive and negative inter-packet delays together,
> >> and sometimes these have different meanings or causes.
> >
> > Both the RFC 3393 and 4689 definitions use absolute values.
> >
> > 3393 gives two methods of measuring jitter. One references RFC 2598, the
> > other references RFC 1889, and both of these absolute values. RFC 3393
> > explicitly mentions the use of absolute values for both methods.
> >
> > Thus, this is not really a meaningful differentiator between the 3393
> > and 4689 jitter definitions.
> >
> > Indeed, it would be very problematic *not* to use absolute numbers.
> > Without them, the mean of a uniform distribution of positive and
> > negative values around some midpoint would give us -- zero.
> >
> > Given Lucien's stated goal of "how to make jitter a simple term for
> > datacenter engineers to understand and to measure" (presumably by
> > representing it as a single number, the mean), and given his examples of
> > 5% and 50% variance, it's unclear how a metric that is typically zero
> > would describe that variance in any simple way.
> 
> I wanted to just give a representation of the jitter variance, the
> example values [5%,50%] were just meant to show how it is significant
> for a user and illustrate my point, but it's *not* the jitter
> definition. The goal is to first measure it correctly and what it means
> and then have a comprehensive optional chart histogram representing the
> jitter variance, this is usually done by packet size as on figure 10 for
> example here:
> 
> http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps11541/ps12581
> /white_paper_c11-716751.pdf
> 
> 
> The *key* to me is to first *explain* simply what jitter means for the
> datacenter engineer which I re-quote here from draft-dcbench-def-00
> 
> 
> 3.2 Discussion
> 
>     Jitter can be measured in different scenarios:
>    -packet to packet delay variation
>    -delta between min and max packet delay variation for all
>     packets sent.
> 
> 3.3 Measurement
> 
>     The jitter MUST be measured when sending packets of the same size.
>     Jitter MUST be measured as packet to packet delay variation and delta
>     between min and max packet delay variation of all packets sent. A
>     histogram MAY be provided as a population of packets measured per
>     latency or latency buckets.
> 
> 
> 
> >
> >> If, for example, they wish to know the overall jitter from a path
> >> with ten DUTs in series, we can estimate that value but we have to
> >> measure the right metric at the start to do it
> >
> > Also unclear, given the bmwg's black-box approach to measurement, why
> > the number of elements within a single SUT would matter.
> >
> > Agreed that the key question here is what constitutes the "right metric"
> > regardless of whether we measure jitter for one box or lots of boxes as
> > a system.
> >
> > dn
> >
> > _______________________________________________
> > bmwg mailing list
> > [email protected]
> > https://www.ietf.org/mailman/listinfo/bmwg
> >
> 
> _______________________________________________
> bmwg mailing list
> [email protected]
> https://www.ietf.org/mailman/listinfo/bmwg