Re: [PATCH RFC v2] nbd: fix I/O hang on dead socket and rate-limit console error
Kusaram Devineni <[email protected]> Thu, 16 Jul 2026 15:07:01 +0530
| Newsgroups | dev.linux.lists.syzbot |
|---|---|
| Message-ID | <[email protected]> |
On 15-07-2026 08:30 pm, syzbot wrote:
> When an NBD device is configured with a timeout of 0, a closed socket can
> lead to a permanent I/O hang. The sequence is as follows: a request is sent
> and marked in-flight, but then the socket becomes dead (e.g., due to a
> connection failure). Since the socket is dead, no reply will ever arrive.
> In nbd_xmit_timeout(), if the configured timeout is 0, the code currently
> only checks if the socket has been replaced by comparing cookies. If the
> cookie still matches, the request timer is unconditionally reset and the
> request stays in-flight forever. This causes tasks to hang indefinitely in
> TASK_UNINTERRUPTIBLE, triggering the hung task detector:
>
> INFO: task udevd:5915 blocked in I/O wait for more than 143 seconds.
> Call Trace:
> <TASK>
> schedule+0x164/0x2b0
> io_schedule+0x7f/0xd0
> folio_wait_bit_common+0x836/0xbc0
> do_read_cache_folio+0x1ac/0x590
> read_part_sector+0xb6/0x2b0
> adfspart_check_POWERTEC+0x9a/0x7a0
> bdev_disk_changed+0x851/0x17a0
> blkdev_get_whole+0x372/0x510
> bdev_open+0x324/0xd70
> ...
> </TASK>
>
> Fix this by checking nsock->dead in addition to the cookie check in
> nbd_xmit_timeout(). If the socket is dead, the command is requeued.
> Requeuing returns the request to the existing fallback, reconnect, or
> failure policy. For the reported configuration where no reconnect is
> possible, this results in the request being properly failed, which
> terminates the hung operation.
>
> A secondary issue was observed where repeated attempts to connect to an
> already-in-use NBD device cause console spam because the "nbd%d already in
> use" error message in nbd_genl_connect() is not rate-limited. This can
> delay console_unlock() and trigger NMI backtraces. This is fixed by
> changing the pr_err() to pr_err_ratelimited().
>
> Fixes: 2c272542baee ("nbd: requeue command if the soecket is changed")
> Assisted-by: Gemini:gemini-3.1-pro-preview Gemini:gemini-3-flash-preview syzbot
> Reported-by: [email protected]
> Closes: https://syzkaller.appspot.com/bug?extid=82de77d3f217960f087d
> Link: https://syzkaller.appspot.com/ai_job?id=40e56b38-627e-4b49-b7ac-d3a21e77dba2
> To: "Jens Axboe" <[email protected]>
> To: "Josef Bacik" <[email protected]>
> To: <[email protected]>
> To: <[email protected]>
> Cc: <[email protected]>
>
> ---
> v2:
> - Updated the commit description to precisely detail the sequence leading to the I/O hang.
> - Clarified that requeueing returns the request to the existing fallback/reconnect/failure policy.
> - Highlighted the console spam as a secondary issue.
> - Switched to pr_err_ratelimited() for the "already in use" error in nbd_genl_connect().
>
> v1:
> https://lore.kernel.org/all/[email protected]/T/
> ---
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 8f10762e9..eedb1c870 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -523,7 +523,7 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req)
> blk_rq_bytes(req), (req->timeout / HZ) * cmd->retries);
>
> mutex_lock(&nsock->tx_lock);
> - if (cmd->cookie != nsock->cookie) {
> + if (cmd->cookie != nsock->cookie || nsock->dead) {
> nbd_requeue_cmd(cmd);
> mutex_unlock(&nsock->tx_lock);
> mutex_unlock(&cmd->lock);
> @@ -2172,7 +2172,7 @@ static int nbd_genl_connect(struct sk_buff *skb, struct genl_info *info)
> nbd_put(nbd);
> if (index == -1)
> goto again;
> - pr_err("nbd%d already in use\n", index);
> + pr_err_ratelimited("nbd%d already in use\n", index);
> return -EBUSY;
> }
>
>
>
> base-commit: 8cdeaa50eae8dad34885515f62559ee83e7e8dda
#syz upstream
-kusaram