[ippm] Re: [bmwg] Re: I-D Action: draft-gaikwad-llm- benchmarking-terminology-00.txt

Thank you Madhava for working on these drafts and thank you Med for forwarding them.
Note that I added these documents to the list of BMWG related drafts.
Therefore, I want to invite BMWG folks to review and comment this new proposal.

Regards,

Giuseppe

From: madhava gaikwad <[email protected]>
Sent: Wednesday, January 21, 2026 4:07 AM
To: [email protected]
Cc: Benchmarking Methodology Discussion List <[email protected]>; IP Performance Measurement Discussion List <[email protected]>
Subject: [bmwg] Re: I-D Action: draft-gaikwad-llm-benchmarking-terminology-00.txt


Hi Med, all,

Thanks a lot for forwarding this. This terminology draft is part of a three-document set for LLM inference serving benchmarking:

  1.  Terminology: metrics definitions (TTFT, ITL, TPOT, etc.) https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-terminology/
  2.  Methodology: test procedures, workloads, and reporting formats https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-methodology/
  3.  Profiles: SUT boundary definitions for model engines, AI gateways, AI firewalls, and compound systems https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-profiles/

The documents reference each other and are intended to be reviewed together. I will be very happy to get feedback from the WG on whether this work aligns with BMWG's scope as well as present at an upcoming session if there is interest.

Thank you.

Madhava

On Tue, Jan 20, 2026 at 8:53 PM <[email protected]<mailto:[email protected]>> wrote:
Hi all,

Forwarding this document as this may be relevant for BMWG/IPPM, fwiw.

@Madhava, feel free to share your plans for this document.

Cheers,
Med

-----Message d'origine-----
De : [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>>
Envoyé : mardi 20 janvier 2026 11:07
À : [email protected]<mailto:[email protected]>
Objet : I-D Action: draft-gaikwad-llm-benchmarking-terminology-00.txt


Internet-Draft draft-gaikwad-llm-benchmarking-terminology-00.txt is now available.

   Title:   Benchmarking Terminology for Large Language Model Serving
   Author:  Madhava Gaikwad
   Name:    draft-gaikwad-llm-benchmarking-terminology-00.txt
   Pages:   54
   Dates:   2026-01-20

Abstract:

   This document defines terminology for benchmarking the performance of
   Large Language Model (LLM) inference serving systems.  It establishes
   a shared vocabulary for latency, throughput, resource utilization,
   and quality metrics applicable to inference engines, application
   gateways, and compound agentic systems.  This document defines
   terminology only and does not prescribe benchmark methodologies or
   acceptance thresholds.

The IETF datatracker status page for this Internet-Draft is:
https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-terminology/

_______________________________________________
I-D-Announce mailing list -- [email protected]<mailto:[email protected]> To unsubscribe send an email to [email protected]<mailto:[email protected]>
____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

_______________________________________________
ippm mailing list -- [email protected]
To unsubscribe send an email to [email protected]