Re: [PATCH RFC 0/4] nfsd: per-client fair-queue dispatch

On Fri, 05 Jun 2026, Benjamin Coddington wrote:
> On 4 Jun 2026, at 10:54, Chuck Lever wrote:
> > - I'm not a fan of adding administrative controls, and these days, a
> >   module parameter is outdated and way too global. I'd like to see
> >   that removed until we have a clear need (use cases) for tuning
> 
> Agreed - I'll drop the module parameter.  With the reframe below the fair
> path is cheap enough to be always-on.
> 
> > - Having to rely on client identity is going to be difficult to get
> >   right. ... fairness-by-client-address means all clients behind a NAT
> >   get reduced to a single fairness unit.
> 
> Agreed, and avoiding identity entirely is most of the appeal of the
> reframe - it also gets NFSv3, NFSv4.0 and LOCALIO for free, which solves
> the no-session problem you raised.
> 
> > - The observability you proposed ... worth ... making it a part of the
> >   patch series
> 
> Will do - I'll turn the latency-injection measurement into something that
> stays in the series, but it's probably not appropriate for any final
> inclusion.
> 
> > Instead, we could look at the problem as "preventing starvation of
> > any one connection" ...
> > fq_codel's success comes less from its fair queueing than from its
> > sparse-flow optimization; that is, flows with nothing in flight jump
> > the queue and heavy flows share what remains.
> 
> I like this - it dissolves the identity question, and xpt_nr_rqsts already
> gives us the in-flight signal to classify on.
> 
> One wrinkle I'd want to get right first: what a user perceives as
> "interactive" is a command, not a single RPC, and a command is a burst of
> correlated RPCs - loading a web page whose browser cache is on NFS is a
> readdir, then stat/open/read across many files.  Plain sparse-flow grants
> priority for one quantum (~one RPC for us), so only the first RPC of the
> burst jumps; RPCs 2..N find the connection backlogged, demote to the bulk
> tier, and complete at the aggressor's rate.  The leading edge is fast but
> the command isn't.
> 
> Rough numbers, ~650 ops/s pool, interactive on one connection, aggressor on
> 16, a 50-RPC command:
> 
>   today (per-transport FIFO):     50 * 17 / 650  ~= 1.3 s
>   sparse-flow, 1-RPC quantum:     ~= 1.3 s  (only the first RPC saved)
>   whole command prioritized:      50 / 650       ~= 0.08 s
> 
> So to make the command feel interactive the priority allowance has to cover
> the cycle, not one RPC: grant an idle->active connection a budget of ~N RPCs,
> re-granted each time it returns to idle, rather than a single jump.  This
> stays identity-free and connection-count-proof for the same reason your
> version does - a sustained stream never returns to idle, so it collects the
> allowance once and is in the bulk tier forever after, however many
> connections it opens.  A user-paced connection idles between commands and is
> re-granted each time.  (This also naturally covers Daire's
> workstations-over-render-farm case: bursty-then-idle vs sustained, no subnet
> config.)
> 
> That turns the one open parameter into "how many RPCs is an interactive
> cycle" (N).  I'd rather not make it an admin knob either:

There is another parameter: how much priority boost do you give
"interactive" tasks.  You cannot let interactive completely dominate
batch, else a sufficiently large number of interactive clients can
starve the batch clients.

> 
>   - a generous fixed default - over-provisioning costs almost nothing, since
>     a sustained stream never idles to collect it, and under-provisioning just
>     lets the tail of an unusually large command fall back to bulk;
>   - measure it - track per-connection burst size between idle gaps;
>   - borrow the v4.1 slot count as a per-connection hint (v3 would need a
>     fallback).
> 
> Does a burst-sized allowance fit how you were picturing the sparse tier, or
> were you thinking the single-RPC jump was enough and the rest should just
> share bulk?  That's the main fork I see.
> 
> I'll rework the series around this - always-on, no module parameter, no
> client identity, sparse/bulk tiers keyed on xpt_nr_rqsts, measurement
> included.  I also owe new numbers: my current harness runs every connection
> at a fixed window, so the "interactive" client is actually backlogged -
> the wrong shape to show any of this.  The metric that matters is
> command-completion time versus the aggressor's connection count.
> 
> Thanks for the reframe - it's a much better-shaped problem.

It seems to be a completely different problem, but maybe I misunderstood
your original problem statement.
You seemed to start out focused on nconnect and wanting fairness between
clients with nconnect=1 and those with nconnect=16.  Is this not what
you really want?

The "interactive" frame can only work for clients that are single-user
or few-users.  A non-trivial multi-user system could easily start
looking like a batch system, at least in bursts, and this could give
users surprising changes in latency.

It might make sense to add an "interactive" export flag so that some
clients get a boost.  Manual configuration is best avoided, but not
always possible (which is why we even have export options).

Ideally the client would determine what is "interactive" and use v4.1
sessions to put "interactive" on a different session to "batch".  Then
it should be easier for interactive sessions to get an appropriate
share.


Thanks,
NeilBrown


> 
> Ben
>