[ippm] Re: [bmwg] Re: Progress so far -- Re: Revisit ing BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsear ch

Hi Carsten,

what’s the test purpose?


  *   Minimum Latency / no load
  *   Latency@maximum throughput?

If it is the latter, udp speed test is designed to measure max throughput without causing permanent congestion https://datatracker.ietf.org/doc/draft-ietf-ippm-capacity-protocol/.

Regards,

Ruediger

Von: Qin Wu <[email protected]>
Gesendet: Dienstag, 18. November 2025 02:04
An: Giuseppe Fioccola <[email protected]>; Carsten Rossenhoevel <[email protected]>; Gábor LENCSE <[email protected]>; [email protected]
Cc: [email protected]; [email protected]; [email protected]
Betreff: [ippm] Re: [bmwg] Re: Progress so far -- Re: Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch

Interesting discussion on those cross border topic,
@Carsten I am wondering whether liaison statement can be exchanged between O-RAN and IETF at some time point on such topic.

-Qin
发件人: Giuseppe Fioccola
发送时间: 2025年11月18日 0:03
收件人: Carsten Rossenhoevel <[email protected]<mailto:[email protected]>>; Gábor LENCSE <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>
抄送: [email protected]<mailto:[email protected]>; Qin Wu <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]>
主题: RE: [bmwg] Re: Progress so far -- Re: Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch

(adding IPPM since this topic might also be relevant for the ongoing discussion on the BMWG/IPPM synergies)

Hi Carsten,

Thank you for raising this point.
The use of IPPM related methods (like TWAMP, STAMP,…) can definitely be considered for the renewed benchmarking tests in BMWG.
I would include it to the table that Gabor is preparing.

Regards,

Giuseppe

From: Carsten Rossenhoevel <[email protected]<mailto:[email protected]>>
Sent: Saturday, November 15, 2025 8:55 PM
To: Gábor LENCSE <[email protected]<mailto:[email protected]>>; Giuseppe Fioccola <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>
Subject: RE: [bmwg] Re: Progress so far -- Re: Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch

Hi Gabor, Giuseppe,

Thank you for your important initiative to renovate the legacy performance benchmarking RFCs!  We have already started this process in BMWG with RFC 9411 on network security benchmarking, which obsoletes RFC 3511.

It would be very helpful to brainstorm with BMWG participants how to accurately test latency in IP networks today, using IETF RFCs, and to explore any improvements that could be made to either upgrade the methodology, provide clear guidance, or eliminate outdated options.

We are currently proposing a test methodology for end-to-end Ethernet/IP latency testing in radio access networks (RANs) to the O-RAN Alliance.  I hope that our suggestion based on TWAMP will be adopted.  Some members of the O-RAN Alliance were confused about the statements regarding latency testing in RFC 2544, which is still considered an important base standard.  It would be awesome if all the legacy BMWG documents could be reviewed and aligned/updated.

Thank you!
Carsten

From: Gábor LENCSE <[email protected]<mailto:[email protected]>>
Sent: November 13, 2025 20:03
To: Giuseppe Fioccola <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>
Subject: [bmwg] Re: Progress so far -- Re: Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch


Hi Giuseppe,

Thank you very much for your reply. Please see my answers inline.
On 11/13/2025 12:24 PM, Giuseppe Fioccola wrote:
Hi Gabor,
Thank you for your initial analysis.
As a contributor, I have some comments:
1.       I agree that RFC 4814 should update RFC 2544 since it discusses hash and stuffing for testing, but I’m not sure about RFC 8219. Is there something in RFC 8219 (BM for IPv6 Transition Technologies) that has relevance for the general benchmarking guidelines?

I believe YES.

For example, RFC 2544 defined a rather simple latency measurement procedure.  It uses a single tagged frame sent after 60s in a 120s long stream for latency measurement. (Please refer to Section 26.2 of RFC 2544.) On the other hand, RFC 8219 defined a much better one: it uses at least 500 tagged frames sent after 60s in a 120s long stream for latency measurement. Please see the details here: https://datatracker.ietf.org/doc/html/rfc8219#section-7.2 Thus it can give much better characterizations of the latency conditions.

In addition to that, RFC 8219 also uses Packet Delay Variation and Inter Packet Delay Variation to give further insights.

IMHO, if these improvements were deemed to be necessary to characterize the performance IPv6 transition technologies, then they could also be useful when performing simple IPv4 and IPv6 packet forwarding performance measurements, too.

What do you think?

2.       Another document that could also be considered is RFC 7640 on Traffic Management Benchmarking.
3.       I would also add draft-ietf-bmwg-mlrsearch to the list of the possible updates to RFC 2544.
4.       I suggest to include draft-lencse-bmwg-multiple-ip-addresses too. Even though it is still individual, I think it is relevant.

Yes, I would like to consider them, too.

So far, I did only a very small portion of the planned task and I wanted to check, if there is interest in BMWG to tackle with this topic. If yes, then I am happy to continue the effort. The lion's share of the work is still left.

Best regards,

Gábor



Regards,

Giuseppe


From: Gábor LENCSE <[email protected]><mailto:[email protected]>
Sent: Tuesday, November 11, 2025 9:59 AM
To: Giuseppe Fioccola <[email protected]><mailto:[email protected]>; [email protected]<mailto:[email protected]>
Cc: [email protected]<mailto:[email protected]>
Subject: Progress so far -- Re: [bmwg] Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch


Hi BMWG,

I started to systematically review  all documents produced by BMWG. I collected my notes to an Excel sheet having the following column headings:
- Document (RFC / I-D)
- Full Title
- Main Contributions
- Potential Issues (omissions, contradictions, etc.)
- It refers to
- It is (formally) updated by
- It (perhaps) should be updated by

Do you consider theses aspects relevant?

What else should be considered?

I copy here the part of my Excel sheet that I have already filled in. I am not sure, how it will go through the mailing list:
Document (RFC / I-D)
Full Title
Main Contributions
Potential Issues (omissions, contradictions, etc.)
It refers to
It is (formally) updated by
It (perhaps) should be updated by
RFC 1212
Benchmarking Terminology for Network Interconnection Devices
Defines the following terms:
Back-to-back
Bridge
bridge/router
Constant Load
Data link frame size
Frame Loss Rate
Inter Frame Gap
Latency
Link Speed Mismatch
MTU-mismatch behavior
Overhead behavior
Overloaded behavior
Policy based filtering
Restart behavior
Router
Single frame behavior
Throughput
-
RFC 1944
Benchmarking Methodology for Network Interconnect Devices
-- Obsoleted by RFC 2544
RFC 1242
RFC 2285
Benchmarking Terminology for LAN Switching Devices
Defines further terms, theoretically regarding LAN switching, but several of them is of general interest (e.g. DUT/SUT, bidirectional traffic, etc.)
RFC 1242, RFC 1944
RFC 2432
Terminology for IP Multicast Benchmarking
Defines further terms regarding IP multicast benchmarking.
Introduces the two major classes of documents: terminology and methodology.
Also addresses encapsulation and decapsulation performance.
IPv4 specific, but it is natural.
RFC 1242, RFC 2285
RFC 2544
Defines all important circumstances including test setup, conditions, measurement procedures, frame formats, etc.
uses fixed frame format (fixed IP addresses and port numbers)
the Latency measurement procedure only uses single tagged frame.
RFC 1242
RFC 9004, RFC 6201, RFC 6815
RFC 4814!, RFC 8219?
RFC 2647
Benchmarking Terminology for Firewall Performance
RFC 1242, RFC 2285

Do you consider this review useful?

Best regards,

Gábor


On 4/15/2025 10:26 AM, Giuseppe Fioccola wrote:
Hi Gabor,
Thank you for sharing these considerations.
I think it is worth discussing further in the WG. A possible approach could be to collect all these possible updates to RFC 2544 in a brand new draft, which would aim to explicitly update RFC 2544. It might also be a short document since some potential updates have already been published as RFCs. What do you think?

Regards,

Giuseppe

From: Gábor LENCSE <[email protected]><mailto:[email protected]>
Sent: Friday, April 11, 2025 5:39 PM
To: [email protected]<mailto:[email protected]>
Subject: [bmwg] Revisiting BMWG RFCs -- Re: Re: AD Review of draft-ietf-bmwg-mlrsearch

Hi Med,

As a future work, the WG can revisit some of the reference RFCs it produced. +20 years is sufficiently enough to learn from deployments/experience and make some recommendations about the various specs. That effort can then formally discuss 2544 vs MLRS and sketch relevant recommendations. Hope to see volunteers to help the WG explore that path.
I think it would be worth doing so. Such an overview and discussing potential updates, changes, etc. could be useful.

For example, IMHO, RFC 4814 should have updated RFC 2544, but it did not happen. When I implemented my RFC 8219 compliant SIIT tester called siitperf [1], I followed the test frame format defined in RFC 2544 using fixed port numbers [2]. After finishing it, Al Morton drew my attention to the usage of pseudorandom port numbers recommended by RFC 4814. Then, I modified siitperf to support pseudorandom port numbers [3]. However, it would have been a more straightforward way to design it so rather than later modifying it.

Another thing is that, for example, RFC 8219 redefined the latency measurement procedure of RFC 2544, but only a few people know about this, IMHO, better procedure (those who are interested in IPv6 transition technologies).

So, I think it would be a good idea to revisit BMWG RFCs and discuss some similar questions. I would be willing to contribute to such an effort.

Best regards,

Gábor

[1] G. Lencse, "Siitperf: an RFC 8219 compliant SIIT and stateful NAT64/NAT44 tester", https://github.com/lencsegabor/siitperf

[2] G. Lencse, "Design and Implementation of a Software Tester for Benchmarking Stateless NAT64 Gateways", IEICE Transactions on Communications, vol. E104-B, no.2, pp. 128-140. February 1, 2021. DOI: 10.1587/transcom.2019EBN0010, available: http://doi.org/10.1587/transcom.2019EBN0010

[3] G. Lencse, "Adding RFC 4814 Random Port Feature to Siitperf: Design, Implementation and Performance Estimation", International Journal of Advances in Telecommunications, Electrotechnics, Signals and Systems, vol 9, no 3, pp. 18-26, 2020, DOI: 10.11601/ijates.v9i3.291, available: https://doi.org/10.11601/ijates.v9i3.291

_______________________________________________
ippm mailing list -- [email protected]
To unsubscribe send an email to [email protected]