I-D Action: draft-calabria-bmwg-ai-fabric-inference-bench-04.txt

Internet-Draft draft-calabria-bmwg-ai-fabric-inference-bench-04.txt is now
available.

   Title:   Benchmarking Methodology for AI Inference Serving Network Fabrics
   Authors: Fernando Calabria
            Carlos Pignataro
            Qin Wu
            Giuseppe Fioccola
            Sowjanya Reddy
   Name:    draft-calabria-bmwg-ai-fabric-inference-bench-04.txt
   Pages:   49
   Dates:   2026-08-12

Abstract:

   This document defines benchmarking terminology, methodologies, and
   Key Performance Indicators (KPIs) for evaluating Ethernet-based AI
   inference serving network fabrics.  As Large Language Model (LLM)
   inference deployments scale to disaggregated prefill/decode
   architectures spanning hundreds or thousands of accelerators (GPUs/
   XPUs), the interconnect fabric determines Time to First Token (TTFT),
   Inter-Token Latency (ITL), and aggregate throughput in tokens per
   second (TPS).  This document establishes vendor-independent,
   reproducible test procedures for benchmarking fabric-level
   performance under realistic AI inference workloads.

   Coverage includes RDMA-based KV cache transfer between disaggregated
   prefill and decode workers, Mixture-of-Experts (MoE) expert
   parallelism AllToAll communication, request routing and load
   balancing for inference serving, congestion management under bursty
   inference traffic patterns, and scale/soak testing.  The methodology
   enables direct comparison across NIC transport stacks (RoCEv2 and
   UET) and fabric architectures.

   This document is a companion to the AI training fabric benchmarking
   methodology, which addresses training workloads.

The IETF datatracker status page for this Internet-Draft is:
https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-inference-bench/

There is also an HTML version available at:
https://www.ietf.org/archive/id/draft-calabria-bmwg-ai-fabric-inference-bench-04.html

A diff from the previous version is available at:
https://author-tools.ietf.org/iddiff?url2=draft-calabria-bmwg-ai-fabric-inference-bench-04

Internet-Drafts are also available by rsync at:
rsync.ietf.org::internet-drafts


_______________________________________________
I-D-Announce mailing list -- [email protected]
To unsubscribe send an email to [email protected]