[Bug 296936] [fusefs] write() to a file opened with O_APPEND fails with EBADF

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296936

            Bug ID: 296936
           Summary: [fusefs] write() to a file opened with O_APPEND fails
                    with EBADF
           Product: Base System
           Version: 15.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: kern
          Assignee: [email protected]
          Reporter: [email protected]

Description
***********

On a fusefs mount, a file opened with O_APPEND can be opened successfully, but
the first write to that descriptor fails with errno 9 (EBADF) and the data is
lost.
A plain O_WRONLY (create/truncate) write to the same file works normally.
The failure seems to be in the kernel fusefs driver and is independent of both
the FUSE daemon and the underlying storage: it reproduces with any passthrough
daemon over any local filesystem (ZFS, tmpfs, p9fs, ...).
It breaks every append-mode workflow: shell >>, tee -a, log writers, editors
that append, etc.


Testing Environment
*******************

FreeBSD 15.1-RELEASE-p1 GENERIC arm64
fusefs-libs3-3.18.1
sysutils/fusefs-bindfs-1.18.3   (used only as a trivial passthrough daemon)


Steps to reproduce (as root; bindfs is just a minimal passthrough)
******************************************************************

pkg install -y fusefs-bindfs
kldload fusefs
mkdir -p /tmp/lower /tmp/upper
bindfs /tmp/lower /tmp/upper

printf 'first\n'  >  /tmp/upper/f     # O_WRONLY|O_CREAT|O_TRUNC  -> OK
printf 'second\n' >> /tmp/upper/f     # O_WRONLY|O_APPEND         -> fails

truss of the append shows the open succeeding and the write failing:

openat(AT_FDCWD,"/tmp/upper/f",O_WRONLY|O_APPEND|O_CREAT,0666) = 3 (0x3)
write(3,"second\n",7)                 ERR#9 'Bad file descriptor'


Expected result
***************

The append succeeds and f contains:

first
second


Actual result
*************

write returns EBADF; the appended data is discarded. f still contains only
first.


Analysis
********

The append is serviced through the buffered (bio) write path.
In sys/fs/fuse/fuse_io.c, fuse_io_strategy() resolves the write file handle
with a pid of 0:

/* We don't know the true pid when we're dealing with the cache */
pid_t pid = 0;
...
error = fuse_filehandle_getrw(vp, fflag, &fufh, cred, pid);
if (bp->b_iocmd == BIO_READ && error == EBADF) {
        /* read-modify-write on a cached O_WRONLY file: retry */
        error = fuse_filehandle_get(vp, FWRITE, &fufh, cred, pid);
}
if (error) {
        printf("FUSE: strategy: filehandles are closed\n");
        bp->b_ioflags |= BIO_ERROR;
        ...
}

For a BIO_WRITE there is no fallback equivalent to the BIO_READ retry, so when
fuse_filehandle_getrw() cannot match a write handle (here because of pid == 0)
the strategy routine sets BIO_ERROR and EBADF propagates to userland.

Routing the append through the direct path (fuse_write_directbackend(), which
handles IO_APPEND via uio_setoffset(uio, filesize)) avoids the problem, but
there is currently no way to force it: per-filehandle direct_io is broken (bug
#293088), the -o direct_io mount option is rejected by libfuse, and
vfs.fusefs.data_cache_mode = 0, 1, and 2 all reproduce the failure.


Notes
*****

- Reproduces regardless of the lower filesystem (verified on ZFS, tmpfs, and
p9fs).
- vfs.fusefs.data_cache_mode = 0 / 1 / 2 all fail identically.
- -o default_permissions does not change the behavior.
- Likely the same root cause as bug #236379 (cached write path lacks the true
pid); a working per-fd direct_io path (bug #293088) would also avoid it.
- I have discovered this bug while using FreeBSD + fusefs-bindfs
- This bug report was co-written with the aid of "Claude Opus 4.8"

-- 
You are receiving this mail because:
You are the assignee for the bug.