Re: zfs questions

> On 2. Mar 2026, at 17:16, Takashi YAMAMOTO <[email protected]> wrote:
> 
> hi,
> 
> On Mon, Mar 2, 2026 at 8:19 PM J. Hannken-Illjes <[email protected]> wrote:
>> 
>>> On 1. Mar 2026, at 15:13, Takashi YAMAMOTO <[email protected]> wrote:
>>> 
>>> On Fri, Feb 20, 2026 at 5:09 PM Takashi YAMAMOTO <[email protected]> wrote:
>>>> 
>>>> hi,
>>>> 
>>>> recently i have used netbsd zfs and have a few questions.
>>>> 
>>>> * what's the purpose of zfs_zget_cleaner/VN_RELE_CLEANER?
>>>> i have read the explanation in
>>>> https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=59885 .
>>>> but i don't understand why we want to avoid the normal zfs_zget in
>>>> the first place.
>>>> for me, it seems working w/o these special versions of functions:
>>>> https://github.com/yamt/netbsd-src/commit/438ec29437465e0b7e2b831d4452159f83cad0cf
>>>> (i will attach the same patch to this mail for those who don't like github.)
>>> 
>>> this version had a few deadlock issues.
>>> 
>>> i will attach a fixed version i'm currently testing.
>>> 
>>>> 
>>>> * is anyone using zfs in netbsd seriously? :-)
>>>> 
>>>> * does anyone have a plan to update the codebase to recent openzfs?
>>> <a.diff>
>> 
>> This patch makes the situation better but it fires assertions during unmount.
>> 
>> Running some stress tests on a 16-core amd64 vm with DEBUG+LOCKDEBUG and
>> kern.maxvnodes=1105 unmount crashes as it tries to load a vnode during unmount:
>> 
>> 
>> panic: kernel diagnostic assertion "mp == NULL || (mp->mnt_iflag & IMNT_UNMOUNT) == 0 || vp->v_tag == VT_VFS" failed: file "src/sys/kern/vfs_mount.c", line 545
>> 
>> #12  vfs_insmntque (vp=0xffffb751d4324a80, mp=0xffffb751cf2e2000) at src/sys/kern/vfs_mount.c:545
>> #13  vcache_get (mp=0xffffb751cf2e2000, key=key@entry=0xffffc3070a334bc8, key_len=key_len@entry=8, vpp=vpp@entry=0xffffc3070a334bd8) at src/sys/kern/vfs_vnode.c:1666
>> #14  zfs_zget (zfsvfs=zfsvfs@entry=0xffffc3002f7ba000, obj_num=<optimized out>, obj_num@entry=24865, zpp=zpp@entry=0xffffc3070a334c20) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_znode.c:1287
>> #15  zfs_get_data (arg=0xffffc3002f7ba000, lr=0xffffc30048cd41d8, buf=0xffffc30048cd4298 "", zio=0xffffb751e587e840) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vnops.c:1500
>> #16  zil_lwb_commit (zilog=<optimized out>, itx=0xffffb7517fb6f5c0, lwb=0xffffb751bffa25f8) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zil.c:1144
>> #17 zil_commit_writer (zilog=0xffffb751e664d540) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zil.c:1556
>> #18 zil_commit (zilog=0xffffb751e664d540, foid=<optimized out>) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zil.c:1638
>> #19  zil_commit (zilog=0xffffb751e664d540, foid=0) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zil.c:1620
>> #20 zil_close (zilog=0xffffb751e664d540) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zil.c:1856
>> #21  zfsvfs_teardown (zfsvfs=zfsvfs@entry=0xffffc3002f7ba000, unmounting=unmounting@entry=1) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c:2288
>> #22  zfs_umount (vfsp=0xffffb751cf2e2000, fflag=524288) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c:2445
>> #23  VFS_UNMOUNT (mp=mp@entry=0xffffb751cf2e2000, a=a@entry=524288) at src/sys/kern/vfs_subr.c:1526
>> #24  dounmount (mp=mp@entry=0xffffb751cf2e2000, flags=524288, l=l@entry=0xffffb751e855d800) at src/sys/kern/vfs_mount.c:980
>> #25  sys_unmount (l=0xffffb751e855d800, uap=0xffffc3070a335000, retval=<optimized out>) at src/sys/kern/vfs_syscalls.c:728
>> #26  sy_call (sy=0xffffffff8210a050 <sysent+528>, l=0xffffb751e855d800, uap=0xffffc3070a335000, rval=0xffffc3070a334fb0) at src/sys/sys/syscallvar.h:65
>> #27 sy_invoke (sy=0xffffffff8210a050 <sysent+528>, l=0xffffb751e855d800, uap=0xffffc3070a335000, rval=0xffffc3070a334fb0, code=22) at src/sys/sys/syscallvar.h:94
>> #28  syscall (frame=0xffffc3070a335000) at src/sys/arch/x86/x86/syscall.c:137
>> #29  handle_syscall ()
>> 
>> Here we have mp != NULL and mp->mnt_iflag == 0x122 (IMNT_MPSAFE | IMNT_NCLOOKUP | IMNT_UNMOUNT)
> 
> it's an interesting crash. thank you.
> i tried to reproduce it by myself. but no luck.
> the attached patch (b.diff) would fix it.

Running vflush() until all vnodes are flushed works.  I used the attached and functinal equivalent diff and
my load tests ran for 6 hours without problem.

Allowing to attach new vnodes until the mount is IMNT_GONE looks ok.

Not sure we need to assert TAILQ_EMPTY(&mp->mnt_vnodelist), the "dangling vnode" panic below should catch it.

> well, honestly speaking, i don't understand why zil_close commits the
> zil in the first place.
> my impression is that it should not be necessary because we are
> syncing txg for unmount anyway.
> what do you think?

No idea -- its been much too long I looked deeper int ZFS internals.

>> Disabling this assertion zfs unmount fails to destroy a list as it is not empty:
>> 
>> 
>> panic: solaris assert: list->list_head.list_next == node, file: src/sys/../external/cddl/osnet/dist/uts/common/os/list.c, line: 86
>> 
>> #12  list_destroy (list=list@entry=0xffff940059153040) at src/sys/../external/cddl/osnet/dist/uts/common/os/list.c:86
>> #13  zfsvfs_free (zfsvfs=0xffff940059152000) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c:1439
>> #14  zfs_freevfs (vfsp=0xffff859a2e81d000) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c:2782
>> #15 zfs_umount (vfsp=0xffff859a2e81d000, fflag=<optimized out>) at src/sys/../external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c:2471
>> #16  VFS_UNMOUNT (mp=mp@entry=0xffff859a2e81d000, a=a@entry=524288) at src/sys/kern/vfs_subr.c:1526
>> #17  dounmount (mp=mp@entry=0xffff859a2e81d000, flags=524288, l=l@entry=0xffff859a6a698400) at src/sys/kern/vfs_mount.c:980
>> #18  sys_unmount (l=0xffff859a6a698400, uap=0xffff94070afdc000, retval=<optimized out>) at src/sys/kern/vfs_syscalls.c:728
>> #19  sy_call (sy=0xffffffff8210a050 <sysent+528>, l=0xffff859a6a698400, uap=0xffff94070afdc000, rval=0xffff94070afdbfb0) at src/sys/sys/syscallvar.h:65
>> #20  sy_invoke (sy=0xffffffff8210a050 <sysent+528>, l=0xffff859a6a698400, uap=0xffff94070afdc000, rval=0xffff94070afdbfb0, code=22) at src/sys/sys/syscallvar.h:94
>> #21 syscall (frame=0xffff94070afdc000) at src/sys/arch/x86/x86/syscall.c:137
>> #22  handle_syscall ()
>> 
>> where *list is (list_t *) 0xffff940059153040:
>> 
>> {
>>  list_size = 240,
>>  list_offset = 200,
>>  list_head = {
>>    list_next = 0xffff859a07007ab8,
>>    list_prev = 0xffff8599c322bea8
>>  }
>> }
>> 
>> --
>> J. Hannken-Illjes - [email protected]
> <b.diff>

--
J. Hannken-Illjes - [email protected]

diff -r fd9b63dd9e3f external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c
--- external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c
+++ external/cddl/osnet/dist/uts/common/fs/zfs/zfs_vfsops.c
@@ -2370,6 +2370,8 @@ zfs_umount(vfs_t *vfsp, int fflag)
 #endif
 #ifdef __NetBSD__
 	cred_t *cr = CRED();
+	struct vnode_iterator *marker;
+	vnode_t *vp;
 #endif
 
 	ret = secpolicy_fs_unmount(cr, vfsp);
@@ -2412,12 +2414,27 @@ zfs_umount(vfs_t *vfsp, int fflag)
 	 */
 #ifdef __FreeBSD_kernel__
 	ret = vflush(vfsp, 0, (fflag & MS_FORCE) ? FORCECLOSE : 0, td);
+	if (ret != 0)
+		return (ret);
 #endif
 #ifdef __NetBSD__
-	ret = vflush(vfsp, NULL, (fflag & MS_FORCE) ? FORCECLOSE : 0);
+	/*
+	 * we loop here because zil_commit can bring some vnodes
+	 * back to mnt_vnodelist via zfs_get_data.
+	 */
+	vfs_vnode_iterator_init(vfsp, &marker);
+	while ((vp = vfs_vnode_iterator_next(marker, NULL, NULL))) {
+		VN_RELE(vp);
+		vfs_vnode_iterator_destroy(marker);
+		ret = vflush(vfsp, NULL, (fflag & MS_FORCE) ? FORCECLOSE : 0);
+		if (ret != 0)
+			return (ret);
+		if (zfsvfs->z_log)
+			zil_commit(zfsvfs->z_log, 0);
+		vfs_vnode_iterator_init(vfsp, &marker);
+	}
+	vfs_vnode_iterator_destroy(marker);
 #endif
-	if (ret != 0)
-		return (ret);
 
 #ifdef illumos
 	if (!(fflag & MS_FORCE)) {