Re: [PATCH] svcrdma: Cap Read sink allocations at PAGE_ALLOC_COSTLY_ORDER

On Sat, Jun 6, 2026, at 1:35 PM, Jonathan Flynn wrote:
> I tested the PAGE_ALLOC_COSTLY_ORDER change on the same setup.
> Unfortunately, it did not improve the regression. Throughput was slightly
> worse than the previous GFP_NOWAIT test, measuring 25.4 GiB/s.
>
> Current results are:
> Original regressed build: ~30.3 GiB/s
> GFP_NOWAIT build: ~31.0 GiB/s
> PAGE_ALLOC_COSTLY_ORDER: 25.4 GiB/s
> Commit reverted: ~73.9 GiB/s
>
> I added the results to the shared bundle. (including flamegraph)
>
> The GFP_NOWAIT and the Original Commit flamegraphs are nearly identical.
> The dominant stack being:
> svc_recv()
> -> svc_rdma_build_read_segment_contig()
> -> alloc_pages_noprof()
> -> get_page_from_freelist()
> -> rmqueue_buddy()
>
> The PAGE_ALLOC_COSTLY_ORDER flamegraph is different. Time spent under
> alloc_pages_noprof() is reduced, but the reduction does not translate into
> improved throughput.
>
> The following percentages were observed:
>                                                    Original     GFP_NOWAIT
> COSTLY_ORDER
> svc_recv()                                 76.09%      75.99%
> 78.44%
> alloc_pages_noprof()             58.07%      57.99%               40.29%
> folios_put_refs()                        7.15%        7.19%
> 16.06%
> svc_rdma_read_complete()    7.18%        7.21%               16.08%
>
> In other words, the PAGE_ALLOC_COSTLY_ORDER change reduces time spent in
> the allocation path, but a larger fraction of CPU time then appears under
> svc_rdma_read_complete() and folios_put_refs(), while overall throughput
> decreases further.

The two failed fixes demonstrate that the current folio allocator is
not up to the task -- the problem appears to be on the release side,
where the individual pages have to be merged back into an order-4
compound page. I don't yet see a straightforward way to make it work.

Since we're right up against v7.1-rc7, I've added a patch to nfsd-next
to revert 18755b8c2f24 -- it will get pulled back into 7.1.y as soon as
the v7.2 merge window closes in three weeks.


-- 
Chuck Lever