Re: [PATCH RFC 0/4] nfsd: per-client fair-queue dispatch

On 4 Jun 2026, at 10:54, Chuck Lever wrote:
> - I'm not a fan of adding administrative controls, and these days, a
>   module parameter is outdated and way too global. I'd like to see
>   that removed until we have a clear need (use cases) for tuning

Agreed - I'll drop the module parameter.  With the reframe below the fair
path is cheap enough to be always-on.

> - Having to rely on client identity is going to be difficult to get
>   right. ... fairness-by-client-address means all clients behind a NAT
>   get reduced to a single fairness unit.

Agreed, and avoiding identity entirely is most of the appeal of the
reframe - it also gets NFSv3, NFSv4.0 and LOCALIO for free, which solves
the no-session problem you raised.

> - The observability you proposed ... worth ... making it a part of the
>   patch series

Will do - I'll turn the latency-injection measurement into something that
stays in the series, but it's probably not appropriate for any final
inclusion.

> Instead, we could look at the problem as "preventing starvation of
> any one connection" ...
> fq_codel's success comes less from its fair queueing than from its
> sparse-flow optimization; that is, flows with nothing in flight jump
> the queue and heavy flows share what remains.

I like this - it dissolves the identity question, and xpt_nr_rqsts already
gives us the in-flight signal to classify on.

One wrinkle I'd want to get right first: what a user perceives as
"interactive" is a command, not a single RPC, and a command is a burst of
correlated RPCs - loading a web page whose browser cache is on NFS is a
readdir, then stat/open/read across many files.  Plain sparse-flow grants
priority for one quantum (~one RPC for us), so only the first RPC of the
burst jumps; RPCs 2..N find the connection backlogged, demote to the bulk
tier, and complete at the aggressor's rate.  The leading edge is fast but
the command isn't.

Rough numbers, ~650 ops/s pool, interactive on one connection, aggressor on
16, a 50-RPC command:

  today (per-transport FIFO):     50 * 17 / 650  ~= 1.3 s
  sparse-flow, 1-RPC quantum:     ~= 1.3 s  (only the first RPC saved)
  whole command prioritized:      50 / 650       ~= 0.08 s

So to make the command feel interactive the priority allowance has to cover
the cycle, not one RPC: grant an idle->active connection a budget of ~N RPCs,
re-granted each time it returns to idle, rather than a single jump.  This
stays identity-free and connection-count-proof for the same reason your
version does - a sustained stream never returns to idle, so it collects the
allowance once and is in the bulk tier forever after, however many
connections it opens.  A user-paced connection idles between commands and is
re-granted each time.  (This also naturally covers Daire's
workstations-over-render-farm case: bursty-then-idle vs sustained, no subnet
config.)

That turns the one open parameter into "how many RPCs is an interactive
cycle" (N).  I'd rather not make it an admin knob either:

  - a generous fixed default - over-provisioning costs almost nothing, since
    a sustained stream never idles to collect it, and under-provisioning just
    lets the tail of an unusually large command fall back to bulk;
  - measure it - track per-connection burst size between idle gaps;
  - borrow the v4.1 slot count as a per-connection hint (v3 would need a
    fallback).

Does a burst-sized allowance fit how you were picturing the sparse tier, or
were you thinking the single-RPC jump was enough and the rest should just
share bulk?  That's the main fork I see.

I'll rework the series around this - always-on, no module parameter, no
client identity, sparse/bulk tiers keyed on xpt_nr_rqsts, measurement
included.  I also owe new numbers: my current harness runs every connection
at a fixed window, so the "interactive" client is actually backlogged -
the wrong shape to show any of this.  The metric that matters is
command-completion time versus the aggressor's connection count.

Thanks for the reframe - it's a much better-shaped problem.

Ben