Re: I-D IPv6 transition technologies benchmarking

Hi Marius and Bhuvan,

Every few years we have a discussion of "Jitter" on BMWG list.
There are many ways to measure this metric, and how you measure
should be determined by the way you will use the results.
IOW, there doesn't seem to be a generic answer - some
specific need determines your next step.  Some user applications
are less sensitive to "jitter" than others, so they do not
influence the choice of metric - others do.

To that end, I usually cite this RFC:
http://tools.ietf.org/html/rfc5481
and let folks think about it.

regards,
Al

From: bmwg [mailto:[email protected]] On Behalf Of Marius Georgescu
Sent: Monday, December 08, 2014 3:16 AM
To: 'Bhuvaneswaran Vengainathan'; [email protected]
Subject: Re: [bmwg] I-D IPv6 transition technologies benchmarking

Hello Bhuvan,

Thank you very much for taking the time to review the draft. All the comments are pertinent and I will try my best to address them in the next version of the draft. To make sure I understood your comments correctly, here are some inline short answers. I started my answers with MG: …


1.       Section 1.1 - May need to consider tunneling mechanism (e.g., GRE) as one of the transport technology for sending IPvX/Y frames over non-supported core networks.

-          MG: GRE is only one of the encapsulation technologies. There are many other suitable examples: 6in4, 4in6, LISP… My idea is to classify them generically using the 4 mentioned categories. In this case a network device using GRE could fit in the Encapsulation-based technologies and use the associated test setup.

2.       Section 3.2 Encapsulation/Translation based transition technologies - I think the MTU settings can be recommended for the interfaces between CE and PE DUTs.

-          MG: I agree that a MTU recommendation can be very useful for the reader, especially in the case of encapsulation. I will include it in the next version

3.       Section 4 Test Traffic - This section limits the test traffic to unicast. How about handling the Multicast traffic?

-          MG: The main scope of the methodology is unicast traffic. I think treating multicast traffic can   overcomplicate the test setups.  Maybe I should make the scope clearer.

4.       Section 6.0 - Benchmarking Tests. The routing mode (static or dynamic) between edges may influence the benchmarking test results. Should it be explicitly mentioned?

-          MG: I agree the routing can influence the results. I will make clear recommendations in the  next version.

5.       Section 6.0 - How about including a test for Jitter?

-          MG: I was thinking about including jitter as well :) I will definitely include it in the next version. Aside from jitter, I was thinking about some operational benchmarks as well.

6.       Section 7.2- Reporting Format - Should the reporting format also include a column for Traffic Type (e.g., TCP or UDP)? Also it is recommended to capture Traffic Type as part of Section 4.

-          MG: One of the things  I understood from the IETF91 meeting: TCP benchmarking  is a “don’t go there” zone because of the unpredictability of TCP flows. As Scott Bradner and Al Morton very well pointed out, the methodology should result in a reasonable basis for comparison. Consequently, I plan to note clearly that the test traffic SHOULD be UDP.

7.       Appendix A.1 - The Frame size and fps calculation explained here is for UDP frames? For TCP, the frame size/fps depends on TCP window sizes’ too.

-          MG: It’s clear I did not explain very well the formula. I added RFC5180 as a reference for the formula, but I will  add some more explanatory text. The 20 bytes in the formula is the sum of the Ethernet preamble (8 bytes) and the inter-frame gap (12 bytes); it only covers Layer 2 recommendations. As I explained for the previous comment, UDP will be the recommended transport layer protocol.


Best regards,
Marius

From: Bhuvaneswaran Vengainathan [mailto:[email protected]]
Sent: Monday, December 08, 2014 2:23 PM
To: 'Marius Georgescu'; [email protected]<mailto:[email protected]>
Subject: RE: [bmwg] I-D IPv6 transition technologies benchmarking

Hi Marius,


I gave a quick look at the draft. The draft is interesting and seems to be in the right direction.
Here are some if my comments on this draft.

1.       Section 1.1 - May need to consider tunneling mechanism (e.g., GRE) as one of the transport technology for sending IPvX/Y frames over non-supported core networks.

2.       Section 3.2 Encapsulation/Translation based transition technologies - I think the MTU settings can be recommended for the interfaces between CE and PE DUTs.

3.       Section 4 Test Traffic - This section limits the test traffic to unicast. How about handling the Multicast traffic?

4.       Section 6.0 - Benchmarking Tests. The routing mode (static or dynamic) between edges may influence the benchmarking test results. Should it be explicitly mentioned?

5.       Section 6.0 - How about including a test for Jitter?

6.       Section 7.2- Reporting Format - Should the reporting format also include a column for Traffic Type (e.g., TCP or UDP)? Also it is recommended to capture Traffic Type as part of Section 4.

7.       Appendix A.1 - The Frame size and fps calculation explained here is for UDP frames? For TCP, the frame size/fps depends on TCP window sizes’ too.

Best Regards,


Bhuvan | Cloud and SDN Solutions
Phone: +91 44 4567 2222 x242
Mobile: +91 988 428 6693
[cid:[email protected]]
www.veryxtech.com<http://www.veryxtech.com/>

From: bmwg [mailto:[email protected]] On Behalf Of Marius Georgescu
Sent: Wednesday, September 24, 2014 9:38 AM
To: [email protected]<mailto:[email protected]>
Subject: [bmwg] I-D IPv6 transition technologies benchmarking

Dear BMWG Members,

My name is Marius Georgescu and I am a graduate student at Nara Institute of Science and Technology (NAIST) in Japan. My main research focus is IPv6 transition technologies benchmarking.  Recently I have been working on an Internet draft which can be found here:
http://www.ietf.org/id/draft-georgescu-ipv6-transition-tech-benchmarking-00.txt

I hope to present the draft in the next BMWG meeting, in IETF91. In the meantime, here is the summary of the draft.

The draft is complementary to the recommendations of RFC2544 and RFC5180, focusing on IPv6 transition technologies benchmarking. It includes a tentative classification of transition technologies and proposes associated test setups. It also proposes a method to acknowledge the frame size overhead and how it should be considered when calculating the maximum theoretical frame rates, in order to avoid exceeding the bandwidth of the employed media.

The  draft also includes a tentative benchmark for scalability in the context of network devices.
Scalability is seen as the ability of network devices to accommodate network growth. As network growth can produce performance degradation, quantifying this degradation can offer insights on the scalability of a certain network devices.
The network growth  can be generated by the tester by creating multiple network flows. After measuring the performance in the context of multiple flows, the performance degradation can be expressed as relative change between the multiple flow results  and the single flow results.

As the draft is very close to the scope of the BMWG, I was hoping it can become a BMWG draft at some point. Any comments and suggestion are welcome. Thank you very much.

Best regards,

Marius Georgescu 　(マリウス ジョルジェスク)
Internet Engineering Laboratory
Nara Institute of Science and Technology
mailto: [email protected]<mailto:[email protected]>
IPv6NET Project: http://www.ipv6net.ro/

_______________________________________________
bmwg mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/bmwg