Re: [PATCH] io_uring/io-wq: avoid repeated task_work scans during teardown

> From: "Gabriel Krisman Bertazi"<[email protected]>
> Date:  Wed, May 20, 2026, 21:45
> Subject:  Re: [PATCH] io_uring/io-wq: avoid repeated task_work scans during teardown
> To: "Fengnan Chang"<[email protected]>
> Cc: <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>, <[email protected]>
> "Fengnan Chang" <[email protected]> writes:
> 
> > We hit hard-lockup reports from iou-wrk threads stuck in
> 
> It seems like a soft-lockup instead no?  From your description,
> eventually it solves itself, the task is just uninterruptible while
> contending on the spinlock.
hard-lockup, here is the log:

May 19 11:20:41 n154-134-017 kernel: [  672.229430][  C138] watchdog: CPU138: Watchdog detected hard LOCKUP on cpu 138
May 19 11:20:41 n154-134-017 kernel: [  672.229435][  C138] Modules linked in: binfmt_misc(E) msr(E) nft_compat(E) x_tables(E) ip_set_hash_net(E) ip_set(E) nf_tables(E) nfnetlink(E) bonding(E) i10nm_edac(E) skx_edac_common(E) nfit(E) edac_core(E) intel_rapl_msr(E) intel_rapl_common(E) intel_uncore_frequency(E) intel_uncore_frequency_common(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) coretemp(E) btrfs(E) ast(E) qat_4xxx(E) snd_pcm(E) kvm_intel(E) drm_client_lib(E) libblake2b(E) tpm_tis(E) snd_timer(E) drm_shmem_helper(E) tpm_tis_core(E) intel_qat(E) snd(E) cxl_acpi(E) pmt_telemetry(E) kvm(E) crc8(E) drm_kms_helper(E) iTCO_wdt(E) tpm(E) raid6_pq(E) cxl_pmem(E) soundcore(E) isst_if_mmio(E) pmt_discovery(E) isst_if_mbox_pci(E) authenc(E) irqbypass(E) aesni_intel(E) gf128mul(E) rapl(E) intel_cstate(E) intel_uncore(E) libnvdimm(E) mei_me(E) pmt_class(E) rng_core(E) dax_hmem(E) pcspkr(E) efi_pstore(E) drm(E) zstd_compress(E) xor(E) i2c_i801(E) idxd(E) bnxt_en(E) mei(E) isst_if_common(E) intel_vsec(E) idxd_bus(E) i2c_smbus(E) i2c_ismt(E) wmi(E) aead(E) i2c_algo_bit(E)
May 19 11:20:41 n154-134-017 kernel: [  672.229527][  C138]  acpi_power_meter(E) button(E) joydev(E) evdev(E) hid_generic(E) usbhid(E) hid(E) acpi_ipmi(E) ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) efivarfs(E) autofs4(E) xhci_pci(E) xhci_hcd(E) nvme(E) usbcore(E) usb_common(E) nvme_core(E)
May 19 11:20:41 n154-134-017 kernel: [  672.229553][  C138] irq event stamp: 47578
May 19 11:20:41 n154-134-017 kernel: [  672.229555][  C138] hardirqs last  enabled at (47577): [<ffffffffa4c49fc9>] _raw_spin_unlock_irqrestore+0x39/0x60
May 19 11:20:41 n154-134-017 kernel: [  672.229561][  C138] hardirqs last disabled at (47578): [<ffffffffa4c49cd7>] _raw_spin_lock_irqsave+0x67/0x70
May 19 11:20:41 n154-134-017 kernel: [  672.229566][  C138] softirqs last  enabled at (45744): [<ffffffffa21f88c7>] handle_softirqs+0x577/0x840
May 19 11:20:41 n154-134-017 kernel: [  672.229571][  C138] softirqs last disabled at (45739): [<ffffffffa21f9726>] irq_exit_rcu+0xe6/0x280
May 19 11:20:41 n154-134-017 kernel: [  672.229576][  C138] CPU: 138 UID: 0 PID: 34918 Comm: iowq-exit-stres Kdump: loaded Tainted: G S          EL      7.1.0-rc3-debug-iomap+ #99 PREEMPT(lazy) 
May 19 11:20:41 n154-134-017 kernel: [  672.229583][  C138] Tainted: [S]=CPU_OUT_OF_SPEC, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
May 19 11:20:41 n154-134-017 kernel: [  672.229585][  C138] Hardware name: Inventec G220-B6/Yichun MLB, BIOS 03.01.02.04.02 10/30/2023
May 19 11:20:41 n154-134-017 kernel: [  672.229587][  C138] RIP: 0010:native_queued_spin_lock_slowpath+0x55d/0xc90
May 19 11:20:41 n154-134-017 kernel: [  672.229592][  C138] Code: c0 75 3f 48 8b 8d 30 ff ff ff 48 b8 00 00 00 00 00 fc ff df 48 89 ca 83 e1 07 48 c1 ea 03 49 89 cd 48 01 c2 41 83 c5 03 f3 90 <0f> b6 02 41 38 c5 7c 08 84 c0 0f 85 9c 05 00 00 41 8b 47 08 85 c0
May 19 11:20:41 n154-134-017 kernel: [  672.229596][  C138] RSP: 0018:ff110085599f7598 EFLAGS: 00000046
May 19 11:20:41 n154-134-017 kernel: [  672.229599][  C138] RAX: 0000000000000000 RBX: ff110085599f7658 RCX: 0000000000000000
May 19 11:20:41 n154-134-017 kernel: [  672.229602][  C138] RDX: ffe21c0bcffe7ab9 RSI: 1ffffffff4b0d54e RDI: ffffffffa586aa70
May 19 11:20:41 n154-134-017 kernel: [  672.229604][  C138] RBP: ff110085599f7680 R08: ffe21c10ab340157 R09: ffe21c10ab340157
May 19 11:20:41 n154-134-017 kernel: [  672.229607][  C138] R10: ffe21c10ab340156 R11: ff11008559a00ab3 R12: 00000000022c0000
May 19 11:20:41 n154-134-017 kernel: [  672.229610][  C138] R13: 0000000000000003 R14: ff11008559a00ab0 R15: ff11005e7ff3d5c0
May 19 11:20:41 n154-134-017 kernel: [  672.229613][  C138] FS:  00007fa81d48e500(0000) GS:ff11005ed9544000(0000) knlGS:0000000000000000
May 19 11:20:41 n154-134-017 kernel: [  672.229615][  C138] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 19 11:20:41 n154-134-017 kernel: [  672.229618][  C138] CR2: 0000000032f88000 CR3: 0000008432d9f003 CR4: 0000000000f73ef0
May 19 11:20:41 n154-134-017 kernel: [  672.229621][  C138] PKRU: 55555554
May 19 11:20:41 n154-134-017 kernel: [  672.229622][  C138] Call Trace:
May 19 11:20:41 n154-134-017 kernel: [  672.229624][  C138]  <TASK>
May 19 11:20:41 n154-134-017 kernel: [  672.229627][  C138]  ? __pfx_native_queued_spin_lock_slowpath+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229632][  C138]  ? ring_buffer_unlock_commit+0x130/0x570
May 19 11:20:41 n154-134-017 kernel: [  672.229638][  C138]  ? io_worker_cancel_cb+0x100/0x100
May 19 11:20:41 n154-134-017 kernel: [  672.229647][  C138]  do_raw_spin_lock+0x1e5/0x2a0
May 19 11:20:41 n154-134-017 kernel: [  672.229652][  C138]  ? __pfx_do_raw_spin_lock+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229656][  C138]  ? __pfx_function_trace_call+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229661][  C138]  ? _raw_spin_lock_irqsave+0x67/0x70
May 19 11:20:41 n154-134-017 kernel: [  672.229666][  C138]  ? __pfx_io_task_work_match+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229671][  C138]  _raw_spin_lock_irqsave+0x52/0x70
May 19 11:20:41 n154-134-017 kernel: [  672.229674][  C138]  ? task_work_cancel_match+0xf4/0x260
May 19 11:20:41 n154-134-017 kernel: [  672.229680][  C138]  task_work_cancel_match+0xf4/0x260
May 19 11:20:41 n154-134-017 kernel: [  672.229686][  C138]  ? __pfx_task_work_cancel_match+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229691][  C138]  ? __pfx_io_task_work_match+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229696][  C138]  ? task_work_cancel_match+0x9/0x260
May 19 11:20:41 n154-134-017 kernel: [  672.229700][  C138]  ? find_held_lock+0x35/0xb0
May 19 11:20:41 n154-134-017 kernel: [  672.229707][  C138]  io_wq_cancel_tw_create+0x82/0xc0
May 19 11:20:41 n154-134-017 kernel: [  672.229713][  C138]  io_wq_put_and_exit+0xdd/0x770
May 19 11:20:41 n154-134-017 kernel: [  672.229717][  C138]  ? xa_find_after+0x1c1/0x330
May 19 11:20:41 n154-134-017 kernel: [  672.229723][  C138]  ? __pfx_io_wq_put_and_exit+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229729][  C138]  ? io_uring_del_tctx_node+0x2ce/0x3c0
May 19 11:20:41 n154-134-017 kernel: [  672.229737][  C138]  io_uring_clean_tctx+0x125/0x1b0
May 19 11:20:41 n154-134-017 kernel: [  672.229742][  C138]  ? __pfx_io_uring_clean_tctx+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229746][  C138]  ? lock_is_held_type+0xa7/0x120
May 19 11:20:41 n154-134-017 kernel: [  672.229751][  C138]  ? __kasan_check_write+0x18/0x20
May 19 11:20:41 n154-134-017 kernel: [  672.229758][  C138]  io_uring_cancel_generic+0x5cb/0xe70
May 19 11:20:41 n154-134-017 kernel: [  672.229764][  C138]  ? __lock_acquire+0xc71/0x1ea0
May 19 11:20:41 n154-134-017 kernel: [  672.229769][  C138]  ? __pfx_io_uring_cancel_generic+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229776][  C138]  ? __kasan_check_write+0x18/0x20
May 19 11:20:41 n154-134-017 kernel: [  672.229780][  C138]  ? do_raw_spin_lock+0x130/0x2a0
May 19 11:20:41 n154-134-017 kernel: [  672.229785][  C138]  ? __pfx_autoremove_wake_function+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229791][  C138]  ? _raw_spin_unlock_irq+0x2b/0x50
May 19 11:20:41 n154-134-017 kernel: [  672.229794][  C138]  ? trace_hardirqs_on+0x2e/0x1a0
May 19 11:20:41 n154-134-017 kernel: [  672.229800][  C138]  __io_uring_cancel+0x1f/0x30
May 19 11:20:41 n154-134-017 kernel: [  672.229804][  C138]  do_exit+0x366/0x34e0
May 19 11:20:41 n154-134-017 kernel: [  672.229810][  C138]  ? lock_is_held_type+0xa7/0x120
May 19 11:20:41 n154-134-017 kernel: [  672.229814][  C138]  ? lock_is_held_type+0xa7/0x120
May 19 11:20:41 n154-134-017 kernel: [  672.229819][  C138]  ? __pfx_do_exit+0x10/0x10
May 19 11:20:41 n154-134-017 kernel: [  672.229824][  C138]  ? _raw_spin_unlock_irq+0x2b/0x50
May 19 11:20:41 n154-134-017 kernel: [  672.229827][  C138]  ? trace_hardirqs_on+0x2e/0x1a0
May 19 11:20:41 n154-134-017 kernel: [  672.229831][  C138]  ? __kasan_check_read+0x15/0x20
May 19 11:20:41 n154-134-017 kernel: [  672.229837][  C138]  do_group_exit+0xbf/0x260
> 
> > + */
> > +struct callback_head *
> > +task_work_cancel_match_all(struct task_struct *task,
> > +                           bool (*match)(struct callback_head *, void *data),
> > +                           void *data)
> > +{
> > +        struct callback_head **pprev = &task->task_works;
> > +        struct callback_head *work, *next;
> > +        struct callback_head *head = NULL, **tail = &head;
> > +        unsigned long flags;
> > +
> > +        if (likely(!task_work_pending(task)))
> > +                return NULL;
> > +
> > +        raw_spin_lock_irqsave(&task->pi_lock, flags);
> > +        work = READ_ONCE(*pprev);
> > +        while (work && work != &work_exited) {
> > +                next = READ_ONCE(work->next);
> > +                if (!match(work, data)) {
> > +                        pprev = &work->next;
> > +                        work = next;
> > +                        continue;
> > +                }
> > +
> > +                if (!try_cmpxchg(pprev, &work, next))
> > +                        continue;
> 
> 
> IIUC, you could ignore the cmpxchg here because the following loop
> iteration on the caller would catch it and retry.  In this case, it no
> retry in io_wq_cancel_tw_create, which looks weird.  Did I miss something?

There is no need retry in io_wq_cancel_tw_create.
In the main teardown path, IO_WQ_BIT_EXIT has already been set by the
time io_wq_cancel_tw_create()  is called. 
As a result, the workqueue is already in the exit state, and normal worker
creation is no longer allowed to proceed.
However, due to a concurrency window, a small number of late create_work
items may still get queued after exit has begun. 
These late arrivals are not handled by an outer retry loop; instead, they are
cleaned up by the post-add exit check in io_queue_worker_create() , which
calls io_wq_cancel_tw_create() again if needed.
Therefore, calling task_work_cancel_match_all() only once does not miss any
Work that must be canceled during the overall teardown process.
> 
> > +
> > +                work->next = NULL;
> > +                *tail = work;
> > +                tail = &work->next;
> > +                work = next;
> > +        }
> > +        raw_spin_unlock_irqrestore(&task->pi_lock, flags);
> > +
> > +        return head;
> > +}
> > +
> >  static bool task_work_func_match(struct callback_head *cb, void *data)
> >  {
> >          return cb->func == data;
> 
> -- 
> Gabriel Krisman Bertazi
>