[PATCH] io_uring/tctx: set ->io_uring before publishing the tctx node
Lim HyeonJun <[email protected]>
| Newsgroups | org.kernel.vger.io-uring |
|---|---|
| Message-ID | <[email protected]> |
io_register_iowq_max_workers() walks ctx->tctx_list under ctx->tctx_lock
and dereferences each node's task->io_uring without a NULL check:
list_for_each_entry(node, &ctx->tctx_list, ctx_node) {
tctx = node->task->io_uring;
if (WARN_ON_ONCE(!tctx->io_wq))
continue;
...
}
__io_uring_add_tctx_node() installs the node into ctx->tctx_list (via
io_tctx_install_node(), which does the list_add() under tctx_lock) and
only assigns current->io_uring = tctx afterwards. A task doing its first
io_uring operation on a shared ring therefore has a window in which its
node is already visible on ctx->tctx_list while node->task->io_uring is
still NULL. A concurrent IORING_REGISTER_IOWQ_MAX_WORKERS on the same
ring reads that NULL and dereferences tctx->io_wq:
KASAN: null-ptr-deref in range [0x0000000000000018-0x000000000000001f]
RIP: io_register_iowq_max_workers io_uring/register.c:423
Publish current->io_uring = tctx before installing the node, so any node
visible on ctx->tctx_list always has a valid task->io_uring. The
tctx_lock taken in io_tctx_install_node() orders this store before the
node becomes visible to other iterators. On the install/limits failure
paths the freshly allocated tctx is freed, so clear current->io_uring
there as well to avoid leaving a dangling pointer.
The bug reproduces on an SMP+KASAN build with a plain (non-SQPOLL) ring
shared across threads: a stream of fresh threads each do their first
io_uring_enter() while two threads spam IORING_REGISTER_IOWQ_MAX_WORKERS;
it GPFs within seconds.
Fixes: 7880174e1e5e ("io_uring/tctx: clean up __io_uring_add_tctx_node() error handling")
Signed-off-by: Lim HyeonJun <[email protected]>
---
io_uring/tctx.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/io_uring/tctx.c b/io_uring/tctx.c
index 6af62ca9baba..42b219b34aa8 100644
--- a/io_uring/tctx.c
+++ b/io_uring/tctx.c
@@ -139,12 +139,14 @@ static int io_tctx_install_node(struct io_ring_ctx *ctx,
int __io_uring_add_tctx_node(struct io_ring_ctx *ctx)
{
struct io_uring_task *tctx = current->io_uring;
+ bool new_tctx = false;
int ret;
if (unlikely(!tctx)) {
tctx = io_uring_alloc_task_context(current, ctx);
if (IS_ERR(tctx))
return PTR_ERR(tctx);
+ new_tctx = true;
if (data_race(ctx->int_flags) & IO_RING_F_IOWQ_LIMITS_SET) {
unsigned int limits[2];
@@ -168,13 +170,15 @@ int __io_uring_add_tctx_node(struct io_ring_ctx *ctx)
if (tctx->io_wq)
io_wq_set_exit_on_idle(tctx->io_wq, false);
- ret = io_tctx_install_node(ctx, tctx);
- if (!ret) {
+ if (new_tctx)
current->io_uring = tctx;
+
+ ret = io_tctx_install_node(ctx, tctx);
+ if (!ret)
return 0;
- }
- if (!current->io_uring) {
err_free:
+ if (new_tctx) {
+ current->io_uring = NULL;
if (tctx->io_wq) {
io_wq_exit_start(tctx->io_wq);
io_wq_put_and_exit(tctx->io_wq);
--
2.53.0