Re: [PATCH RFC 0/4] nfsd: per-client fair-queue dispatch

On 3 Jun 2026, at 18:52, NeilBrown wrote:

> On Thu, 04 Jun 2026, Benjamin Coddington wrote:

>>   - Mechanism: a fixed bucket hash under a per-pool spinlock taken only
>>     on the opt-in path, versus a lockless or per-flow structure.
>
> I'm not keen on the hash bucket approach, though I can't clearly say
> why.

I don't like the hash collisions.  Maybe that's your unspoken reason.
Unlucky client means I still get random hand-wavy reports.

> I'm also not keen on the opt-in design.  I don't like asking the admin to
> tune performance.  We should always provide optimal performance.
>
> I imagine creating an object which represents a client - using IP
> address or possibly v4 client id.  These may well be located using a
> hash table (rhashtable?), but that is peripheral to the design.
> Each xprt has an associated client which can be changed at any time
> (nfsd could change to a v4.1-client).
>
> Each client has a lwq of xprts and is part of an lwq of clients.
>
> To enqueue an xprt, we grab the client, enqueue to that, then if
> necessary enqueue the client.
>
> To dequeue next, we dequeue the first client, dequeue the first xprt,
> then optionally enqueue the client again.

ah - cool.  I think this is a much better design, the enqueue is lockless
and no bucket sharing.

> So this would be slightly more work than the current (2 dequeues instead
> of 1) but I think that might be acceptable.
>
> clients would be refcounted by xprts and probably rcu-freed.

This is the hard part - the xprt holding ref to client, moving from v3 style
(by ip addr) to v4.1 (by clientid) on session bind means repointing the xprt
to a different client object, which can race if the xprt is enqueued.

I suppose it's a one-time event for each v4.1 xprt, so could just create
another XPRT_ flag for it..

I'll see what I can come up with.

>
>>
>>   - Would a per-client in-flight cap be preferable to proportional fair
>>     queueing?
>
> A per-client cap would only be ok if the admin didn't have to tune it.
> So the client would need to get some sort of feed-back from the server
> so that it knows when it is pushing too hard.  If we were still using
> UDP we could possibly use packet-loss for that feed-back, but we aren't
> and don't want to.
> With v4.1 we can of course use the slot based flow control and I think
> we should if we can agree on a good design.  With v3 I don't think there
> is any way to get the needed feed-back

I need to fairly queue v3 and v4.1 clients at the same time... :/

Appreciate your look at this.

Ben