Re: [PATCH 2/2] block: take i_rwsem for the direct I/O write fallback

On 8/2/26 7:14 AM, Tal Zussman wrote:
> Commit c0e473a0d226 ("block: fix race between set_blocksize and read
> paths") closed a race between set_blocksize() and block device I/O: with
> large sector size support, set_blocksize() can change i_blkbits and the
> mapping's minimum folio order while a concurrent reader still holds a
> folio of the old, smaller order, leading to crashes. In particular, it
> made blkdev_write_iter() wrap buffered writes in inode_lock_shared().
> 
> However, the direct I/O fallback path was missed in that conversion.
> blkdev_write_iter() passes blkdev_buffered_write() as an argument to
> direct_write_fallback() with no lock held. A direct write that completes
> only partially then finishes as a buffered write with no protection.
> 
> This can cause a BUG by racing partial direct writes against
> ioctl(BLKBSZSET). Writer threads issue O_DIRECT pwritev() with a
> two-segment iovec whose second segment is an unreadable PROT_NONE
> mapping. The direct path then writes the first segment, fails to pin the
> second, and returns short, entering the fallback. A second thread keeps
> toggling the second segment's protection so that some fallbacks get past
> fault_in_iov_iter_readable() and reach the page cache, a third thread
> populates the page cache with folios of the current block size via
> pread() and readahead(), and a fourth thread toggles the block size
> between 512 bytes and 64K with BLKBSZSET. The minimum folio order only
> moves with block sizes above PAGE_SIZE, i.e. with
> CONFIG_TRANSPARENT_HUGEPAGE raising BLK_MAX_BLOCK_SIZE to 64K.
> 
> On a CONFIG_DEBUG_VM kernel this yields the following BUG:
> 
>   page dumped because: VM_BUG_ON_FOLIO(folio_order(folio) < mapping_min_folio_order(mapping))
>   kernel BUG at mm/filemap.c:858!
>   Oops: invalid opcode: 0000 [#1] SMP KASAN NOPTI
>   RIP: 0010:__filemap_add_folio+0x860/0x8d0
>   Call Trace:
>    filemap_add_folio+0xc9/0x1f0
>    __filemap_get_folio_mpol+0x240/0x660
>    iomap_write_begin+0xa87/0xd70
>    iomap_file_buffered_write+0x304/0x6a0
>    blkdev_write_iter+0x255/0x510
>    do_iter_readv_writev+0x23d/0x3c0
>    vfs_writev+0x211/0x7d0
>    do_pwritev+0x121/0x190
>    do_syscall_64+0x121/0x630
>    entry_SYSCALL_64_after_hwframe+0x77/0x7f
> 
> The same workload also trips WARN_ON_ONCE(pos >= folio_pos(folio) +
> fsize) in iomap_trim_folio_range().
> 
> Fix this by calling blkdev_buffered_write() in the fallback path under
> inode_lock_shared(), matching the plain buffered-write branch. With the
> fix the same workload runs clean.
> 
> The reproducer used was written by an LLM, and is available at [1].
> 
> [1] https://gist.github.com/tzussman/69d06bc57d42a42989eb038b1b5aeb74
> 
> Fixes: c0e473a0d226 ("block: fix race between set_blocksize and read paths")
> Reported-by: Sashiko <[email protected]>
> Link: https://sashiko.dev/#/patchset/20260730-blk-dontcache-v7-0-3e8e6850068d%40columbia.edu?part=5
> Assisted-by: Claude:claude-fable-5
> Signed-off-by: Tal Zussman <[email protected]>
> ---
>  block/fops.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/block/fops.c b/block/fops.c
> index 5b938f673d4f..26711cc239d5 100644
> --- a/block/fops.c
> +++ b/block/fops.c
> @@ -763,9 +763,14 @@ static ssize_t blkdev_write_iter(struct kiocb *iocb, struct iov_iter *from)
>  
>  	if (iocb->ki_flags & IOCB_DIRECT) {
>  		ret = blkdev_direct_write(iocb, from);
> -		if (ret >= 0 && iov_iter_count(from))
> -			ret = direct_write_fallback(iocb, from, ret,
> -					blkdev_buffered_write(iocb, from));
> +		if (ret >= 0 && iov_iter_count(from)) {
> +			ssize_t ret2;
> +
> +			inode_lock_shared(bd_inode);
> +			ret2 = blkdev_buffered_write(iocb, from);
> +			inode_unlock_shared(bd_inode);
> +			ret = direct_write_fallback(iocb, from, ret, ret2);
> +		}
>  	} else {
>  		/*
>  		 * Take i_rwsem and invalidate_lock to avoid racing with
> 

I seem to have stumbled into a Sashiko rabbit hole of issues with IOCB_NOWAIT
and IOCB_ATOMIC here... [1].
I'll wait a day or two and send an update along with some additional fixes
I've spotted.

[1] https://sashiko.dev/#/patchset/20260802-blkdev-fixes-v1-0-a82fc549fd74%40columbia.edu?part=2