[ippm] Re: I-D Action: draft-gaikwad-llm-benchmarking-term inology-00.txt

Hi Med, all,

Thanks a lot for forwarding this. This terminology draft is part of a
three-document set for LLM inference serving benchmarking:

   1. *Terminology: *metrics definitions (TTFT, ITL, TPOT, etc.)
   https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-terminology/
   2. *Methodology: *test procedures, workloads, and reporting formats
   https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-methodology/
   3. *Profiles: *SUT boundary definitions for model engines, AI gateways,
   AI firewalls, and compound systems
   https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-profiles/

The documents reference each other and are intended to be reviewed
together. I will be very happy to get feedback from the WG on whether this
work aligns with BMWG's scope as well as present at an upcoming session if
there is interest.

Thank you.

Madhava

On Tue, Jan 20, 2026 at 8:53 PM <[email protected]> wrote:

> Hi all,
>
> Forwarding this document as this may be relevant for BMWG/IPPM, fwiw.
>
> @Madhava, feel free to share your plans for this document.
>
> Cheers,
> Med
>
> -----Message d'origine-----
> De : [email protected] <[email protected]>
> Envoyé : mardi 20 janvier 2026 11:07
> À : [email protected]
> Objet : I-D Action: draft-gaikwad-llm-benchmarking-terminology-00.txt
>
>
> Internet-Draft draft-gaikwad-llm-benchmarking-terminology-00.txt is now
> available.
>
>    Title:   Benchmarking Terminology for Large Language Model Serving
>    Author:  Madhava Gaikwad
>    Name:    draft-gaikwad-llm-benchmarking-terminology-00.txt
>    Pages:   54
>    Dates:   2026-01-20
>
> Abstract:
>
>    This document defines terminology for benchmarking the performance of
>    Large Language Model (LLM) inference serving systems.  It establishes
>    a shared vocabulary for latency, throughput, resource utilization,
>    and quality metrics applicable to inference engines, application
>    gateways, and compound agentic systems.  This document defines
>    terminology only and does not prescribe benchmark methodologies or
>    acceptance thresholds.
>
> The IETF datatracker status page for this Internet-Draft is:
>
> https://datatracker.ietf.org/doc/draft-gaikwad-llm-benchmarking-terminology/
>
> _______________________________________________
> I-D-Announce mailing list -- [email protected] To unsubscribe send an
> email to [email protected]
>
> ____________________________________________________________________________________________________________
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou
> falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and
> delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been
> modified, changed or falsified.
> Thank you.
>
>

_______________________________________________
ippm mailing list -- [email protected]
To unsubscribe send an email to [email protected]