Re: [PATCH 2/2] io_uring: switch local task_work to a mpscq

On Thu, Jun 11, 2026 at 7:23 PM Jens Axboe <[email protected]> wrote:
>
> On 6/11/26 7:14 PM, Caleb Sander Mateos wrote:
> > This is great stuff! I had also observed these hotspots on a ublk
> > workload. Since incoming ublk requests post task work to the ublk
> > server's io_urings and completed ublk requests post task work to the
> > client's io_urings, there is significant cross-CPU contention on the
> > task work queues.
>
> Glad you like it! Once I post v2 tomorrow, perhaps you can try and run
> some tests with and without and see how it does for you?

Haven't tested v2 yet, but v1 shows a 4% IOPS improvement on a ublk
4-KB read workload. The workload has 8 CPUs (unpaired hypertwins)
running fio with io_uring submitting I/O to the ublk devices and 32
ublk server CPUs (paired hypertwins) servicing the requests, achieving
around 4M IOPS. Both the client and server CPUs look completely busy.
I can see clear reductions in __io_req_task_work_add() and
llist_reverse_order() (now gone) on both sets of CPUs, through the
cache misses popping task work items are now attributed to
__io_run_local_work() instead.

Thanks,
Caleb

>
> >> @@ -185,55 +183,47 @@ void io_req_local_work_add(struct io_kiocb *req, unsigned flags)
> >>
> >>         guard(rcu)();
> >
> > Is the RCU guard still required now that a work list element can't be
> > accessed after the consumer has popped it?
>
> It's actually not. Might need the :
>
>         if (prev == &ctx->work_list.stub) {
>                 io_ctx_mark_taskrun(ctx);
>
> parts to just grab it in there, as lower down we'd still need it. But
> the task_work part itself should not. I'll make that change.
>
> Thanks for the reviews!
>
> --
> Jens Axboe