Re: Lockdep circular dependency: btrfs_swap_activate() calls sysfs_notify()

=E5=9C=A8 2026/7/31 20:27, Christian Borntraeger =E5=86=99=E9=81=93:
> One or more of the following files ( btrfs-swapon-kernfs-repro.sh )=20
> violates IBM policy and all attachment(s) have been removed from the=20
> message.

If the bash script, as an attachment, violates the policy, can you just=20
paste the content as a reply?

As I still have some uncertainty on the LLM explanation.
A concrete reproducer would allow us to do our own experiments and get=20
everything more clear.

Otherwise really appreciated all your reports, no matter if it's=20
generated by LLM or not, as so far your reports are 100% real bugs.

Thanks,
Qu

>=20
> **********************************************************************
> We had the calltrace at the bottom of this mail in our CI logs
> I also attached an AI generated reproducer that triggers this easily.
>=20
> For convenience, here is what AI came up with analysing the log, but
> I would like your take on it. Let me know if you want to see the
> AI proposed fix.
>=20
> ----
>=20
> btrfs_swap_activate() takes the inode's i_mmap_lock for write very
> early (inode.c:10120) and holds it across the entire function; the
> comment there explains the intent, which is to keep mmap writes from
> racing with the delalloc flush and the extent range lock.
>=20
> There are three btrfs_exclop_finish() calls inside that window:
>=20
>  =C2=A0 fs/btrfs/inode.c:10181=C2=A0=C2=A0 error path, swapfile on a rw =
subvolume
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 with an active snapshot
>  =C2=A0 fs/btrfs/inode.c:10202=C2=A0=C2=A0 error path, could not lock th=
e snapshot drew
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 lock
>  =C2=A0 fs/btrfs/inode.c:10399=C2=A0=C2=A0 the common "out:" path -- tak=
en on both
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 success and failure
>=20
> and btrfs_exclop_finish() (fs/btrfs/fs.c:224) ends with an unconditional
> sysfs_notify().=C2=A0 So the offending edge is taken on *every* successf=
ul
> swapon of a btrfs swap file, not only on an error path.
>=20
> The sysfs_notify() itself does nothing but a kernfs lookup plus a poll
> wakeup for userspace watching the "exclusive_operation" attribute.=C2=A0=
 It
> has no dependency whatsoever on i_mmap_lock, or on the inode at all.
>=20
>=20
> Both kernfs_rwsem acquisitions in the cycle are read acquisitions
> ({++++}), so the two of them alone cannot deadlock.=C2=A0 A real hang ne=
eds a
> third task waiting to take kernfs_rwsem for write, because rwsem
> write-fairness makes a later down_read() block behind a queued writer:
>=20
>  =C2=A0 T_swapon:=C2=A0 holds i_mmap_lock(write)
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 blocks in down_read(kernfs_rwsem)=C2=A0=C2=A0 [queued behind T_w]
>  =C2=A0 T_w:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 blocks in down_write(ke=
rnfs_rwsem)=C2=A0 [waiting for T_dir]
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 e.g. any sysfs node create/remove -- device hotplug,
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 module load, cgroup or block-device attribute changes
>  =C2=A0 T_dir:=C2=A0=C2=A0=C2=A0=C2=A0 holds kernfs_rwsem(read) in kernf=
s_fop_readdir
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 faults on the user dirent buffer -> mmap_lock
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 -> btrfs_page_mkwrite -> down_read(i_mmap_lock)
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 blocks behind T_swapon's write holder
>=20
>  =C2=A0 -> three-way deadlock.
>=20
> That is a narrow race, which is consistent with this having gone
> unnoticed for years, but every step of it is ordinary system activity.
> The dependency is genuine and worth fixing rather than annotating away.
>=20
> Suggested fix is to get the sysfs_notify() out from under i_mmap_lock
>=20
>=20
> real life log found in our CI:
> ----------------------------
> LOCKDEP_CIRCULAR (suite: tela-distro, case: tests/test_mempig/test_mempi=
g)
> WARNING: possible circular locking dependency detected
> 7.2.0-20260730.rc5.git10.af7a8a7752eb.300.fc44.s390x+debug #1 Not tainte=
d
> ------------------------------------------------------
> swapon/172010 is trying to acquire lock:
> 000002ea80a485a0 (&root->kernfs_rwsem){++++}-{3:3}, at:=20
> kernfs_find_and_get_ns+0x3c/0x80
> but task is already holding lock:
> 000002ebc465d270 (&ei->i_mmap_lock){++++}-{3:3}, at:=20
> btrfs_swap_activate+0x9a/0x1240
> which lock already depends on the new lock.
> the existing dependency chain (in reverse order) is:
> -> #3 (&ei->i_mmap_lock){++++}-{3:3}:
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lock_acquire+0x150/0x3f0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 down_read+0x5a/0x280
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 btrfs_page_mkwrite+0x258/0x870
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 do_page_mkwrite+0x60/0x160
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 do_wp_page+0x128/0x750
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __handle_mm_fault+0x1be/0x590
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 handle_mm_fault+0xa2/0x370
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 do_exception+0x292/0x590
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __do_pgm_check+0x168/0x430
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pgm_check_handler+0x114/0x160
> -> #2 (sb_pagefaults){.+.+}-{0:0}:
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lock_acquire+0x150/0x3f0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 percpu_down_read_internal.constpro=
p.0+0x54/0x120
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 btrfs_page_mkwrite+0xa6/0x870
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 do_page_mkwrite+0x60/0x160
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 do_fault+0x132/0x4a0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __handle_mm_fault+0x1be/0x590
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 handle_mm_fault+0xa2/0x370
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 do_exception+0x1a0/0x590
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __do_pgm_check+0x168/0x430
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 pgm_check_handler+0x114/0x160
> -> #1 (&mm->mmap_lock){++++}-{3:3}:
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lock_acquire+0x150/0x3f0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __might_fault+0x7a/0xa0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 filldir64+0x11c/0x210
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 kernfs_fop_readdir+0x150/0x4c0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 iterate_dir+0xcc/0x2d0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __do_sys_getdents64+0x7a/0x130
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __do_syscall+0x172/0x750
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 system_call+0x72/0x90
> -> #0 (&root->kernfs_rwsem){++++}-{3:3}:
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 check_prev_add+0x160/0xf40
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __lock_acquire+0x12aa/0x15a0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lock_acquire+0x150/0x3f0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 down_read+0x5a/0x280
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 kernfs_find_and_get_ns+0x3c/0x80
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sysfs_notify+0x60/0xc0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 btrfs_swap_activate+0x83c/0x1240
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __do_sys_swapon+0x278/0x9c0
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 __do_syscall+0x172/0x750
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 system_call+0x72/0x90
> other info that might help us debug this:
> Chain exists of:
>  =C2=A0 &root->kernfs_rwsem --> sb_pagefaults --> &ei->i_mmap_lock
>  =C2=A0Possible unsafe locking scenario:
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 CPU0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 CPU1
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ----=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 ----
>  =C2=A0 lock(&ei->i_mmap_lock);
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lock(sb_pagefaults);
>  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 lock(&ei->i_mmap_lock);
>  =C2=A0 rlock(&root->kernfs_rwsem);
>  =C2=A0*** DEADLOCK ***
> 2 locks held by swapon/172010:
>  =C2=A0#0: 000002ebc465d3f0 (&sb->s_type->i_mutex_key#20){++++}-{3:3}, a=
t:=20
> __do_sys_swapon+0x5be/0x9c0
>  =C2=A0#1: 000002ebc465d270 (&ei->i_mmap_lock){++++}-{3:3}, at:=20
> btrfs_swap_activate+0x9a/0x1240
> stack backtrace:
> CPU: 6 UID: 0 PID: 172010 Comm: swapon Not tainted=20
> 7.2.0-20260730.rc5.git10.af7a8a7752eb.300.fc44.s390x+debug #1 PREEMPT
> Hardware name: IBM 8561 T01 701 (z/VM 7.4.0)
> Call Trace:
>  =C2=A0[<000003f7d5ab4e3e>] dump_stack_lvl+0xae/0x108
>  =C2=A0[<000003f7d5bbef24>] print_circular_bug+0x1a4/0x230
>  =C2=A0[<000003f7d5bbf13c>] check_noncircular+0x18c/0x1b0
>  =C2=A0[<000003f7d5bc0510>] check_prev_add+0x160/0xf40
>  =C2=A0[<000003f7d5bc408a>] __lock_acquire+0x12aa/0x15a0
>  =C2=A0[<000003f7d5bc44d0>] lock_acquire+0x150/0x3f0
>  =C2=A0[<000003f7d6c447ca>] down_read+0x5a/0x280
>  =C2=A0[<000003f7d60ad61c>] kernfs_find_and_get_ns+0x3c/0x80
>  =C2=A0[<000003f7d60b3b70>] sysfs_notify+0x60/0xc0
>  =C2=A0[<000003f7d635653c>] btrfs_swap_activate+0x83c/0x1240
>  =C2=A0[<000003f7d5f1d268>] __do_sys_swapon+0x278/0x9c0
>  =C2=A0[<000003f7d6c369a2>] __do_syscall+0x172/0x750
>  =C2=A0[<000003f7d6c4baa2>] system_call+0x72/0x90
> INFO: lockdep is turned off.