Re: [PATCH v3] bcache: fix cached_dev.sb_bio use-after-free and crash
Jens Axboe <[email protected]> Tue, 31 Mar 2026 10:34:19 -0600
| Newsgroups | org.kernel.vger.linux-bcache |
|---|---|
| Message-ID | <[email protected]> |
On 3/31/26 9:46 AM, Coly Li wrote: > On Tue, Mar 31, 2026 at 09:56:18AM +0800, Coly Li wrote: >> On Mon, Mar 30, 2026 at 09:11:53PM +0800, [email protected] wrote: >>> From: Mingzhe Zou <[email protected]> >>> >>> In our production environment, we have received multiple crash reports >>> regarding libceph, which have caught our attention: >>> >>> ``` >>> [6888366.280350] Call Trace: >>> [6888366.280452] blk_update_request+0x14e/0x370 >>> [6888366.280561] blk_mq_end_request+0x1a/0x130 >>> [6888366.280671] rbd_img_handle_request+0x1a0/0x1b0 [rbd] >>> [6888366.280792] rbd_obj_handle_request+0x32/0x40 [rbd] >>> [6888366.280903] __complete_request+0x22/0x70 [libceph] >>> [6888366.281032] osd_dispatch+0x15e/0xb40 [libceph] >>> [6888366.281164] ? inet_recvmsg+0x5b/0xd0 >>> [6888366.281272] ? ceph_tcp_recvmsg+0x6f/0xa0 [libceph] >>> [6888366.281405] ceph_con_process_message+0x79/0x140 [libceph] >>> [6888366.281534] ceph_con_v1_try_read+0x5d7/0xf30 [libceph] >>> [6888366.281661] ceph_con_workfn+0x329/0x680 [libceph] >>> ``` >>> >>> After analyzing the coredump file, we found that the address of dc->sb_bio >>> has been freed. We know that cached_dev is only freed when it is stopped. >>> >>> Since sb_bio is a part of struct cached_dev, rather than an alloc every time. >>> If the device is stopped while writing to the superblock, the released address >>> will be accessed at endio. >>> >>> This patch hopes to wait for sb_write to complete in cached_dev_free. >>> >>> Signed-off-by: Mingzhe Zou <[email protected]> >>> >> >> Hi Mingzhe, >> >> Yeah, this patch is in better shape. Thank for the fix up again. >> >>> --- >>> v2: fix the crash caused by not calling closure_init in v1 >>> v3: pair the down and up of semaphore sb_write_mutex >>> --- >>> drivers/md/bcache/super.c | 8 ++++++++ >>> 1 file changed, 8 insertions(+) >>> >>> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c >>> index 64bb38c95895..97d9adb0bf96 100644 >>> --- a/drivers/md/bcache/super.c >>> +++ b/drivers/md/bcache/super.c >>> @@ -1373,6 +1373,14 @@ static CLOSURE_CALLBACK(cached_dev_free) >>> >>> mutex_unlock(&bch_register_lock); >>> >>> + /* >>> + * Wait for any pending sb_write to complete before free. >>> + * The sb_bio is embedded in struct cached_dev, so we must >>> + * ensure no I/O is in progress. >>> + */ >>> + down(&dc->sb_write_mutex); >>> + up(&dc->sb_write_mutex); >>> + >>> if (dc->sb_disk) >>> folio_put(virt_to_folio(dc->sb_disk)); >>> >> >> The patch itself is good. But the previous one (based on closre_sync()) is >> in Jens block-tree. Let me ask. >> >> Hi Jens, >> >> Should Mingzhe send a incremental patch based on commit b36478a1fece in >> block-7.0 branch of linux-block tree? Or Just use this patch to replace >> the in-linux-block-tree one? >> > > Hi Mingzhe, > > Considering no responds from Jens at this moment, I assume that an incremental > patch againt the already-in-linux-block patch might be more convenient for him. > Then he can simply apply the incremental patch without extra rebase stuffs. > > Can you re-compose a patch against block-7.0 branch of linux-block tree? Yes send a new patch, I'm not going to rebase the tree just to fold in a fix for a patch that's deep in the stack. The fixup doesn't even mention which patch it's fixing either... -- Jens Axboe