[PATCH v7 1/4] elf: Release dl_load_lock before running dlopen constructors (BZ 15686)
[email protected] Mon, 3 Aug 2026 23:03:23 +0300
| Newsgroups | gmane.comp.lib.glibc.alpha |
|---|---|
| Message-ID | <[email protected]> |
From: Artem Proskurnev <[email protected]> This addresses one instance of the long-standing class of deadlocks described in BZ #15686: ELF constructors and destructors invoked by the dynamic loader run with dl_load_lock held, so any code path in those constructors that itself needs dl_load_lock deadlocks. dl_open_worker holds dl_load_lock across the entire _dl_open call, including the call to call_dl_init that runs the new objects' constructors. If one of those constructors spawns a thread whose first access to a thread_local object triggers __cxa_thread_atexit_impl, the new thread blocks trying to acquire dl_load_lock -- which is held by the dlopen thread -- deadlocking the process. The same deadlock arises when the spawned thread calls a function that triggers NSS module loading through _dl_open, or any other code path that needs dl_load_lock. The blocking site is __cxa_thread_atexit_impl at stdlib/cxa_thread_atexit_impl.c. BZ #28357 was a partial fix for the wider BZ #15686 problem: it moved dl_open_worker_begin and _dl_close_worker to the finer-grained dl_load_tls_lock (commit 024a7640ab) and used that new lock in pthread_create and __tls_get_addr. __cxa_thread_atexit_impl, however, still takes dl_load_lock to protect its DSO lookup (_dl_find_dso_for_object) against a racing dlclose, and that path is not covered by the BZ #28357 fix. Moving it to dl_load_tls_lock is not straightforward because _dl_find_dso_for_object walks _ns_loaded, which is protected by dl_load_lock rather than dl_load_tls_lock. This patch takes the alternative approach of releasing dl_load_lock during constructor execution. At the point where call_dl_init runs, the following invariants hold: * All link_map structures for the newly loaded DSO and its dependencies are fully initialized and immutable. * The DSO has l_direct_opencount == 1 (incremented in dl_open_worker_begin), so a concurrent dlclose cannot unload it: _dl_close_worker short-circuits when the count is non-zero. * Implicit dependencies are protected by the l_map_used marking in _dl_close_worker, which transitively marks the l_initfini chain of any map with non-zero l_direct_opencount. * dl_iterate_phdr uses dl_load_write_lock rather than dl_load_lock and is unaffected by the unlock. Other threads calling dl_iterate_phdr during the constructor may observe the DSO before its constructor has run; this is consistent with POSIX, which does not guarantee atomic appearance of dlopen'd objects, and is equivalent to dlsym from inside a constructor observing partially-initialized main-executable symbols. * Recursive dlopen from a constructor re-acquires dl_load_lock normally in _dl_open and proceeds serially. The lock is re-acquired immediately after constructors complete, before add_to_global_update and the lock/unlock pairing expected by _dl_open. Exception safety: call_dl_init is invoked via _dl_catch_exception (NULL, ...) so that lazy binding failures are fatal (the process exits through _dl_fatal_printf); therefore the re-lock is not required on the error path. C++ exceptions thrown from constructors are a separate, pre-existing concern: dl exception handling uses setjmp/longjmp rather than C++ unwinding, so a thrown exception may leave locks in any state regardless of this patch. Releasing the lock is strictly safer than holding it in that case. Minimal reproducer: a DSO whose constructor calls gdk_pixbuf_new_from_file on a system where the glycin image loader is wired in via gdk-pixbuf reaches a sandboxed loader process spawn, which in turn calls std::thread::spawn; the spawned thread's first thread_local access (__cxa_thread_atexit_impl) blocks on dl_load_lock held by the dlopen caller. The same hang reproduces with any constructor that spawns a thread touching thread_local state or triggering NSS module loads. A regression test is added in sysdeps/pthread/tst-create2.c with its DSO in tst-create2mod.c. The DSO constructor spawns a worker thread that calls __cxa_thread_atexit_impl and then joins it; under the pre-fix locking model the join deadlocks and the test framework times out. The test follows the layout of tst-create1 (BZ #28357), which covers the pthread_create leg of the same bug class. Releasing the lock introduces a data race on l_init_called in call_init (elf/dl-init.c): two threads performing concurrent dlopen of the same DSO could both pass the l_init_called check and run the constructor in parallel. This is addressed by adding per-DSO init serialisation using an l_init_once field in struct link_map. call_init uses atomic_compare_and_exchange_bool_acq to claim the right to run the constructor (0 -> 1); the winning thread proceeds, while losing threads block via lll_futex_wait (or __lll_wait on Hurd) until the winner signals completion (1 -> 2) via lll_futex_wake (or __lll_wake). Different DSOs can initialise concurrently because each has its own l_init_once. The CAS / futex_wait / futex_wake dance is only needed when other threads may be waiting, so single-threaded processes skip it via RTLD_SINGLE_THREAD_P - but l_init_once is always set to 2 after the constructor finishes, so a later call_init from another thread (e.g. via a transitive dlopen once the process goes multi-threaded) for a DSO initialised at startup does not wait on a value that was never going to change. Only the futex wake is conditional on RTLD_SINGLE_THREAD_P; without the unconditional store this bug hung intl/tst-gettext4 and tst-gettext5 in the full make check, because ld.so and libc.so initialise single-threaded at startup but later receive transitive call_init from gconv/NSS loads. A second field, l_init_owner, records the kernel TID of the thread that won the l_init_once CAS and is currently running the constructor. It serves two purposes. First, it closes a return-before-completion race: l_init_called is set at the top of call_init, before the constructor runs, so a concurrent caller that arrives while the constructor is still in progress sees l_init_called set and returns immediately -- even though l_init_once is still 1 and the constructor's writes have not yet been published. With l_init_owner, such a caller compares l_init_owner against its own TID; if they differ, it waits on l_init_once via futex until the winner signals completion. Second, l_init_owner lets a recursive call_init that originates from inside the constructor itself (e.g. via a transitive dlopen of a circular dependency) recognise itself and return immediately, instead of waiting for its own constructor and deadlocking. A third field, l_init_pending, closes a narrow race window that l_init_owner alone does not cover. l_init_called is set only inside call_init, which runs after dl_load_lock has been released and after the CAS has been won; a concurrent dlopen caller that takes the already-loaded early-return path (in dl_open_worker_begin when new->l_searchlist.r_list != NULL, or in _dl_open via is_already_fully_open) could observe l_init_called == 0 and l_init_once == 0 and return before the constructor has been scheduled, defeating the wait that l_init_owner was meant to enforce. l_init_pending is set to 1 in dl_open_worker while it still holds dl_load_lock, just before releasing it to run the constructor, and is cleared by call_init once it has won the l_init_once CAS and become the init owner. Both early-return paths then wait on l_init_once whenever (l_init_pending || l_init_called) is set, l_init_once != 2, and l_init_owner differs from the current TID, releasing dl_load_lock around the wait. A second lock must be released around that same wait in dl_open_worker_begin: dl_load_tls_lock. dl_open_worker calls _dl_catch_exception (..., dl_open_worker_begin, ...) while holding dl_load_tls_lock across the worker, so the early-return wait runs with dl_load_tls_lock held. Releasing only dl_load_lock is not enough - the constructor we are waiting for may spawn a thread, and thread creation takes dl_load_tls_lock in allocatestack -> _dl_allocate_tls_init. Holding dl_load_tls_lock across the futex wait re-introduces the very BZ 15686 deadlock the patch fixes for the single-lock case, in the shape the patch was meant to enable: waiter: holds dl_load_tls_lock, waits on l_init_once futex loader: in ctor -> pthread_create -> _dl_allocate_tls_init waits for dl_load_tls_lock The loader's ctor typically joins the spawned thread, so the ctor never completes, l_init_once never reaches 2, and the waiter never wakes. The __tls_get_addr slow path for dynamically loaded TLS also takes dl_load_tls_lock, so a spawned thread's first dynamic TLS access deadlocks the same way. Fix: release dl_load_tls_lock together with dl_load_lock before the futex wait, and re-acquire them afterwards in the canonical nesting order - dl_load_lock first, then dl_load_tls_lock (the same order _dl_open and dl_open_worker take them). The unlock/relock pairs are balanced by recursion count, so dl_open_worker's own dl_load_tls_lock unlock after dl_open_worker_begin returns stays consistent. The wait is safe with both locks released: the link map is pinned by the l_direct_opencount increment at the top of dl_open_worker_begin, and the futex wait touches only l_init_once. Finally, the _dl_open fast path (is_already_fully_open) was tightened to increment l_direct_opencount BEFORE entering the wait loop, matching the existing increment in dl_open_worker_begin. Without the increment first, the loader thread that ran the constructor could complete, dlsym the DSO, dlclose it (driving opencount to 0 and unloading the DSO), and free the link_map while other threads were still blocked in the wait loop - a use-after-free that surfaced in tst-create3 as segfaults in do_lookup_x with a poisoned scope pointer (0x2a2a2a2a2a2a2a2a) on cleanup, after the test had already printed PASS. A second regression test, tst-create3/tst-create3mod, exercises concurrent dlopen of the same DSO with NTHREADS=8 callers and verifies two invariants: 1. The constructor runs EXACTLY ONCE. This catches the case where two threads both think they lost the l_init_once CAS but proceed anyway. 2. No dlopen caller returns before the constructor has finished. The constructor sleeps 200 ms and publishes a "done" magic as its final write; each caller checks the magic immediately after dlopen returns. This catches the l_init_called short-circuit race that l_init_owner closes -- an earlier version of the test that only checked ctor_count == 1 did NOT catch it. A third test, tst-create4/tst-create4mod-a/tst-create4mod-b, verifies that a long-running constructor does not block a concurrent dlopen of an unrelated library. Thread A's constructor blocks on a barrier; once it has started, thread B dlopen's a different library. Before the BZ 15686 fix, thread B would deadlock because dl_load_lock was still held by thread A across the constructor. After the fix, dl_load_lock is released during the constructor, so thread B's dlopen succeeds in parallel. The test is deterministic: it spins until the constructor has confirmed it started, then creates thread B; no probabilistic interleaving is involved. A fourth regression test, tst-create5, checks a subtle regression in the main-executable path of the BZ 15686 fix. The main executable's constructors are run by the startup code rather than by call_init, so an earlier version of this patch left l_init_once at 0 for the executable. A later multi-threaded dlopen(NULL) / __RTLD_OPENEXEC then took the already-loaded early-return path in dl_open_worker_begin, saw l_init_called == 1 but l_init_once != 2, and waited forever for a constructor that would never complete. tst-create5 spawns a worker thread that calls dlopen(NULL, RTLD_NOW); without the fix the call deadlocks and the test-driver times out, while the fix (setting l_init_once = 2 for the executable in call_init, with a futex wake when multi-threaded) makes dlopen(NULL) return immediately. A fifth regression test, tst-create6/tst-create6mod, exercises the dl_load_tls_lock release around the early-return wait described above. It combines the two ingredients that tst-create2 and tst-create3 cover separately: the module constructor spawns and joins a thread (thread creation takes dl_load_tls_lock in _dl_allocate_tls_init), and two threads concurrently dlopen the same DSO. The constructor sleeps 200 ms before pthread_create so the second caller reliably reaches the early-return wait first. Without the dl_load_tls_lock release around the wait, the constructor's pthread_create blocks on it and the test deadlocks; with the fix, both dlopen calls return only after the constructor published its done magic, and the constructor ran exactly once. To aid diagnosis of such applications, a new tunable glibc.rtld.strict_init_order (default 0) reverts to the pre-BZ-15686 locking model: dl_load_lock is held across constructor execution, giving the old strict total order of dlopen calls across threads at the cost of reintroducing the BZ 15686 deadlock. The tunable is intended as a temporary escape hatch, not a long-term solution, and is documented in manual/tunables.texi. It is enabled at run time through GLIBC_TUNABLES, e.g.: GLIBC_TUNABLES=glibc.rtld.strict_init_order=1 ./your-app Because the tunable is evaluated in dl_open_worker on every dlopen, it can be set on a per-process basis without rebuilding, and cleared again once the downstream application has been fixed. Tested on x86_64-linux-gnu. Both directions of the main regression test verified: sysdeps/pthread/tst-create2 deadlocks (times out after 10 s) on the unpatched tree and passes (exit 0) with this patch. tst-create3 covers both the ctor-runs-once invariant and the visibility invariant that l_init_owner + l_init_pending preserve, and ran 15/15 stable once the opencount-before-wait fix was applied to the _dl_open fast path. tst-create4 is the positive counterpart: it demonstrates that the lock release enables concurrent dlopen of independent libraries, with a barrier-based long-running constructor that would deadlock the unpatched tree but passes deterministically after the fix. tst-create5 was checked by building the rest of this patch without the main-executable fix: that intermediate build fails nptl/tst-create5 (timeout), while the full patch passes it. tst-create6 was verified the same way against the dl_load_tls_lock release: an intermediate build with only the dl_load_lock release but without the dl_load_tls_lock release around the early-return wait hangs (test-driver timeout), while the full patch passes it. The real-world trigger was also verified end-to-end with a minimal reproducer: a DSO whose constructor calls gdk_pixbuf_new_from_file on a PNG, reaching the glycin sandbox loader via gdk-pixbuf and spawning a Rust std::thread whose first thread_local access hits __cxa_thread_atexit_impl. Against the unpatched tree the reproducer hangs (timeout 10 s); against the patched tree it loads the image and exits 0. Full glibc test suite (make check): the only difference between the intermediate build (without the main-executable fix) and the final patched tree is nptl/tst-create5, which moves from FAIL to PASS. In the test runs used here the intermediate build produced 6877 PASS / 8 FAIL and the final tree produced 6878 PASS / 7 FAIL. All seven remaining FAILs on the patched tree are environmental (missing capabilities or test-root permissions at install time) and reproduce identically on the intermediate build. The expected output of elf/tst-rtld-list-tunables was updated for the new glibc.rtld.strict_init_order line. Co-authored-by: Alexander Pevzner <[email protected]> Signed-off-by: Artem Proskurnev <[email protected]> Signed-off-by: Alexander Pevzner <[email protected]> --- elf/dl-init.c | 99 ++++++++++++++++++++- elf/dl-open.c | 133 ++++++++++++++++++++++++++++- elf/dl-tunables.list | 6 ++ elf/tst-rtld-list-tunables.exp | 1 + include/link.h | 22 +++++ manual/tunables.texi | 46 ++++++++++ sysdeps/pthread/Makefile | 45 ++++++++++ sysdeps/pthread/tst-create2.c | 67 +++++++++++++++ sysdeps/pthread/tst-create2mod.c | 74 ++++++++++++++++ sysdeps/pthread/tst-create3.c | 112 ++++++++++++++++++++++++ sysdeps/pthread/tst-create3.h | 27 ++++++ sysdeps/pthread/tst-create3mod.c | 68 +++++++++++++++ sysdeps/pthread/tst-create4.c | 100 ++++++++++++++++++++++ sysdeps/pthread/tst-create4.h | 37 ++++++++ sysdeps/pthread/tst-create4mod-a.c | 30 +++++++ sysdeps/pthread/tst-create4mod-b.c | 22 +++++ sysdeps/pthread/tst-create5.c | 63 ++++++++++++++ sysdeps/pthread/tst-create6.c | 111 ++++++++++++++++++++++++ sysdeps/pthread/tst-create6.h | 26 ++++++ sysdeps/pthread/tst-create6mod.c | 72 ++++++++++++++++ 20 files changed, 1155 insertions(+), 6 deletions(-) create mode 100644 sysdeps/pthread/tst-create2.c create mode 100644 sysdeps/pthread/tst-create2mod.c create mode 100644 sysdeps/pthread/tst-create3.c create mode 100644 sysdeps/pthread/tst-create3.h create mode 100644 sysdeps/pthread/tst-create3mod.c create mode 100644 sysdeps/pthread/tst-create4.c create mode 100644 sysdeps/pthread/tst-create4.h create mode 100644 sysdeps/pthread/tst-create4mod-a.c create mode 100644 sysdeps/pthread/tst-create4mod-b.c create mode 100644 sysdeps/pthread/tst-create5.c create mode 100644 sysdeps/pthread/tst-create6.c create mode 100644 sysdeps/pthread/tst-create6.h create mode 100644 sysdeps/pthread/tst-create6mod.c diff --git a/elf/dl-init.c b/elf/dl-init.c index bd85bacdc1..072f71e237 100644 --- a/elf/dl-init.c +++ b/elf/dl-init.c @@ -20,6 +20,25 @@ #include <stddef.h> #include <ldsodefs.h> #include <elf-initfini.h> +#include <tls.h> + +/* Per-DSO once-initialization for constructor execution. + l_init_once is an int used as a low-level lock (LLL): + 0 = uninitialized, 1 = initializing, 2 = initialized. + The lock is always private to the process. */ + +/* Platform-specific wait/wake primitives for once-initialization. */ +#ifdef __linux__ +# define DL_INIT_ONCE_WAIT(futexp, val, private) \ + lll_futex_wait (futexp, val, private) +# define DL_INIT_ONCE_WAKE(futexp, nr, private) \ + lll_futex_wake (futexp, nr, private) +#else +# define DL_INIT_ONCE_WAIT(futexp, val, private) \ + __lll_wait (futexp, val, private) +# define DL_INIT_ONCE_WAKE(futexp, nr, private) \ + __lll_wake (futexp, private) +#endif static void @@ -35,8 +54,31 @@ call_init (struct link_map *l, int argc, char **argv, char **env) assert (l->l_relocated || l->l_type == lt_executable); if (l->l_init_called) - /* This object is all done. */ - return; + { + /* call_init has already been invoked for this map. In a + single-threaded context this means the constructor has run + (or we are inside it via a recursive call). In a multi- + threaded context, distinguish three cases: + + (a) Constructor already completed (l_init_once == 2): done. + (b) Recursive call from the same thread that is currently + running the constructor (l_init_owner == current TID): + return immediately, otherwise we would wait for our own + constructor to finish and deadlock. + (c) Concurrent call from another thread that is currently + running the constructor: wait for completion. Without + this wait, our caller's dlopen() would return before the + constructor finishes - a regression from the pre-BZ-15686 + model where dl_load_lock serialised the whole _dl_open. */ + if (!RTLD_SINGLE_THREAD_P + && atomic_load_acquire (&l->l_init_once) != 2 + && l->l_init_owner != THREAD_GETMEM (THREAD_SELF, tid)) + { + while (atomic_load_acquire (&l->l_init_once) != 2) + DL_INIT_ONCE_WAIT (&l->l_init_once, 1, LLL_PRIVATE); + } + return; + } /* Avoid handling this constructor again in case we have a circular dependency. */ @@ -45,7 +87,49 @@ call_init (struct link_map *l, int argc, char **argv, char **env) /* Check for object which constructors we do not run here. */ if (__builtin_expect (l->l_name[0], 'a') == '\0' && l->l_type == lt_executable) - return; + { + /* The main executable's constructors are run by the startup code, + not here. Nevertheless we must mark the map as fully + initialized so that a later multi-threaded dlopen(NULL) / + __RTLD_OPENEXEC caller does not wait forever on l_init_once + in dl_open_worker. */ + atomic_store_release (&l->l_init_once, 2); + if (!RTLD_SINGLE_THREAD_P) + DL_INIT_ONCE_WAKE (&l->l_init_once, INT_MAX, LLL_PRIVATE); + return; + } + + /* When single-threaded (startup, or dlopen before any threads exist), + run the constructor inline. When multi-threaded, use per-DSO + serialisation: the first caller runs the constructor; any concurrent + caller blocks until it completes. Different DSOs can initialise + concurrently. */ + if (!RTLD_SINGLE_THREAD_P) + { + /* Fast path: already initialized. */ + if (atomic_load_acquire (&l->l_init_once) == 2) + return; + + /* Try to acquire the lock (CAS 0 -> 1). */ + if (atomic_compare_and_exchange_bool_acq (&l->l_init_once, 1, 0) != 0) + { + /* Another thread is initializing. Wait until it finishes. */ + while (atomic_load_acquire (&l->l_init_once) != 2) + DL_INIT_ONCE_WAIT (&l->l_init_once, 1, LLL_PRIVATE); + return; + } + + /* We won the CAS - record our TID so a recursive call_init from + inside the constructor (e.g. via a transitive dlopen) can + recognise itself and return without deadlocking. */ + l->l_init_owner = THREAD_GETMEM (THREAD_SELF, tid); + + /* We are now the init owner; l_init_pending has done its job of + signalling "init is scheduled" to early-return waiters in + dl_open_worker_begin. Clear it so they don't keep spinning + on it after init completes. */ + atomic_store_release (&l->l_init_pending, 0); + } /* Print a debug message if wanted. */ if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_IMPCALLS)) @@ -73,6 +157,15 @@ call_init (struct link_map *l, int argc, char **argv, char **env) for (j = 0; j < jm; ++j) ((dl_init_t) addrs[j]) (argc, argv, env); } + + /* Mark the DSO as fully initialised so that a later call_init from + another thread (which can happen transitively when a new DSO is + loaded that depends on this one) sees l_init_once == 2 and does + not wait. In single-threaded mode there can be no waiters, so + the futex wake is skipped. */ + atomic_store_release (&l->l_init_once, 2); + if (!RTLD_SINGLE_THREAD_P) + DL_INIT_ONCE_WAKE (&l->l_init_once, INT_MAX, LLL_PRIVATE); } diff --git a/elf/dl-open.c b/elf/dl-open.c index 87fcee8b02..9930371867 100644 --- a/elf/dl-open.c +++ b/elf/dl-open.c @@ -37,6 +37,7 @@ #include <libc-early-init.h> #include <gnu/lib-names.h> #include <dl-find_object.h> +#include <dl-tunables.h> #include <dl-dst.h> #include <dl-prop.h> @@ -597,6 +598,64 @@ dl_open_worker_begin (void *a) dlopen (NULL, RTLD_LAZY) call from a constructor of an initially loaded shared object. */ + /* BZ 15686: dl_load_lock is released during the constructor on + the thread that first loaded this DSO, so this thread may have + reached the already-loaded early-return path while that + constructor is still running (or, in a tight race, after the + loader thread released the lock but before it reached + call_init - covered by l_init_pending which is set under the + lock). Without this wait, dlopen would return before the + constructor finished - a regression from the pre-BZ-15686 model + where dl_load_lock serialised the whole _dl_open. + + We only wait; we do not run the ctor ourselves. Per the + comment above, running _dl_init here could expose partially + constructed state to objects that depend on this DSO if this + dlopen call came from inside another ELF constructor. The + loader thread that scheduled the init (signalled by + l_init_pending or l_init_called) will run the ctor when it + reaches call_init. + + Fast path: if l_init_once == 2, ctor already finished. + + Release dl_load_lock before waiting so concurrent dlopen + callers are not blocked. dl_load_tls_lock (held by + dl_open_worker across this call) must be released too: the + constructor we are about to wait for may spawn a thread, and + thread creation takes dl_load_tls_lock in + _dl_allocate_tls_init. Blocking on the futex while still + holding dl_load_tls_lock would deadlock - the ctor would + wait for pthread_create, which would wait for us. + + The locks are re-acquired in the canonical nesting order: + dl_load_lock first, then dl_load_tls_lock (the same order + _dl_open and dl_open_worker take them). */ + if (!RTLD_SINGLE_THREAD_P + && (atomic_load_acquire (&new->l_init_pending) + || new->l_init_called) + && atomic_load_acquire (&new->l_init_once) != 2 + && new->l_init_owner != THREAD_GETMEM (THREAD_SELF, tid)) + { + bool unlock_for_ctor + = TUNABLE_GET (glibc, rtld, strict_init_order, + int32_t, NULL) == 0; + + if (unlock_for_ctor) + { + __rtld_lock_unlock_recursive (GL(dl_load_tls_lock)); + __rtld_lock_unlock_recursive (GL(dl_load_lock)); + } + + while (atomic_load_acquire (&new->l_init_once) != 2) + lll_futex_wait (&new->l_init_once, 1, LLL_PRIVATE); + + if (unlock_for_ctor) + { + __rtld_lock_lock_recursive (GL(dl_load_lock)); + __rtld_lock_lock_recursive (GL(dl_load_tls_lock)); + } + } + return; } @@ -792,11 +851,46 @@ dl_open_worker (void *a) int mode = args->mode; struct link_map *new = args->map; + /* By default, release dl_load_lock so constructors can spawn + threads without deadlocking (e.g. if the new thread's first + thread_local access triggers __cxa_thread_atexit_impl, which + needs dl_load_lock). See BZ 15686. + + The DSO has l_direct_opencount == 1, so a concurrent dlclose + cannot unload it. Per-DSO serialisation is handled in call_init + (dl-init.c) via l_init_once + futex, and the l_init_owner check + in call_init makes a concurrent dlopen caller wait for the + constructor to finish (rather than returning prematurely). + + Setting glibc.rtld.strict_init_order=1 disables the unlock and + reverts to the pre-BZ-15686 model where dl_load_lock is held + across constructor execution. This gives a strict total order + of dlopen calls across threads - useful for diagnosing + applications that implicitly relied on that order, at the cost + of reintroducing the deadlock. */ + bool release_lock_for_ctor + = TUNABLE_GET (glibc, rtld, strict_init_order, int32_t, NULL) == 0; + + /* Signal "init is scheduled" while still holding dl_load_lock, so a + concurrent dlopen caller that takes the early-return path for an + already-loaded DSO knows to wait for the ctor even before + call_init runs. Cleared by call_init after it wins the + l_init_once CAS. See BZ 15686. */ + atomic_store_release (&new->l_init_pending, 1); + + if (release_lock_for_ctor) + __rtld_lock_unlock_recursive (GL(dl_load_lock)); + /* Run the initializer functions of new objects. Temporarily disable the exception handler, so that lazy binding failures are fatal. */ _dl_catch_exception (NULL, call_dl_init, args); + /* Re-acquire dl_load_lock for the final global scope update and + for the lock/unlock pairing expected by _dl_open. */ + if (release_lock_for_ctor) + __rtld_lock_lock_recursive (GL(dl_load_lock)); + /* Now we can make the new map available in the global scope. */ if (mode & RTLD_GLOBAL) add_to_global_update (new); @@ -889,10 +983,43 @@ no more namespaces available for dlmopen()")); args.map = _dl_lookup_map (args.nsid, file); if (is_already_fully_open (args.map, mode)) { - /* We can use the fast path. */ - ++args.map->l_direct_opencount; + struct link_map *map = args.map; + + /* We can use the fast path. Account for our reference BEFORE + entering the wait below: while we wait, dl_load_lock is released, + and another caller that already holds a reference may dlclose. + Without our own increment first, the last such dlclose could + drive l_direct_opencount to 0 and unload the DSO - and us with + it. */ + ++map->l_direct_opencount; + + /* BZ 15686: dl_load_lock is released during the constructor on + the thread that first loaded this DSO, so this thread may have + reached the already-loaded fast path while that constructor is + still running (or, in a tight race, after the loader thread + released the lock but before it reached call_init - covered by + l_init_pending which is set under the lock). Without this + wait, dlopen would return before the constructor finished. + + Same logic as the early-return path in dl_open_worker_begin; + duplicated here because this fast path bypasses the worker + entirely. */ + if (!RTLD_SINGLE_THREAD_P + && (atomic_load_acquire (&map->l_init_pending) + || map->l_init_called) + && atomic_load_acquire (&map->l_init_once) != 2 + && map->l_init_owner != THREAD_GETMEM (THREAD_SELF, tid)) + { + __rtld_lock_unlock_recursive (GL(dl_load_lock)); + + while (atomic_load_acquire (&map->l_init_once) != 2) + lll_futex_wait (&map->l_init_once, 1, LLL_PRIVATE); + + __rtld_lock_lock_recursive (GL(dl_load_lock)); + } + __rtld_lock_unlock_recursive (GL(dl_load_lock)); - return args.map; + return map; } struct dl_exception exception; diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list index 111649f145..5149dbe1ab 100644 --- a/elf/dl-tunables.list +++ b/elf/dl-tunables.list @@ -113,6 +113,12 @@ glibc { maxval: 2 default: 1 } + strict_init_order { + type: INT_32 + minval: 0 + maxval: 1 + default: 0 + } } mem { diff --git a/elf/tst-rtld-list-tunables.exp b/elf/tst-rtld-list-tunables.exp index 9590021f3a..2a4c7a5eda 100644 --- a/elf/tst-rtld-list-tunables.exp +++ b/elf/tst-rtld-list-tunables.exp @@ -15,3 +15,4 @@ glibc.rtld.enable_secure: 0 (min: 0, max: 1) glibc.rtld.execstack: 1 (min: 0, max: 2) glibc.rtld.nns: 0x4 (min: 0x1, max: 0x10) glibc.rtld.optional_static_tls: 0x200 (min: 0x0, max: 0x[f]+) +glibc.rtld.strict_init_order: 0 (min: 0, max: 1) diff --git a/include/link.h b/include/link.h index 8f851d2212..1047f2ab23 100644 --- a/include/link.h +++ b/include/link.h @@ -346,6 +346,28 @@ struct link_map size_t l_relro_size; unsigned long long int l_serial; + + /* Per-DSO once-initialization control for constructor execution. + Used as a low-level lock (LLL): 0 = uninitialized, 1 = initializing, + 2 = initialized. Zero from calloc matches the unlocked state. */ + int l_init_once; + + /* TID of the thread that won the l_init_once CAS (0 -> 1) and is + currently running this DSO's constructor. Lets a recursive + call_init (originating from inside the constructor itself, e.g. + via a transitive dlopen of a circular dependency) distinguish + itself from a concurrent call_init on another thread and return + immediately instead of waiting for itself. Zero from calloc. */ + pid_t l_init_owner; + + /* Set to 1 under dl_load_lock by dl_open_worker_begin just before + releasing the lock to run the constructor. Lets a concurrent + dlopen caller that takes the early-return path for an + already-loaded DSO know that initialisation is scheduled and + worth waiting for, even before call_init has set l_init_called + and won the l_init_once CAS. Cleared to 0 by call_init once it + has won the CAS and become the init owner. Zero from calloc. */ + int l_init_pending; }; #include <dl-relocate-ld.h> diff --git a/manual/tunables.texi b/manual/tunables.texi index 713f669c4c..8d835fe9b4 100644 --- a/manual/tunables.texi +++ b/manual/tunables.texi @@ -476,6 +476,52 @@ can be worked around by setting the tunable to @code{2}, where the stack is always executable. @end deftp +@deftp Tunable glibc.rtld.strict_init_order +Controls whether @theglibc{} retains @code{dl_load_lock} across the +execution of ELF constructors run by @code{dlopen}. + +The default value of @samp{0} releases @code{dl_load_lock} before running +the new objects' constructors and reacquires it afterwards. This prevents +the deadlock described in @uref{https://sourceware.org/bugzilla/show_bug.cgi?id=15686, BZ 15686}: +a constructor that spawns a thread whose first @code{thread_local} access +calls @code{__cxa_thread_atexit_impl} would otherwise block forever on +@code{dl_load_lock} held by the @code{dlopen} caller. Per-DSO +serialisation of constructor execution (via @code{l_init_once} in +@file{elf/dl-init.c}) preserves the ELF guarantee that a DSO's +dependencies are initialised before the DSO itself, and concurrent +@code{dlopen} callers wait for the in-progress constructor to finish. +This default does not change any behaviour required by POSIX or the ELF +specification; it only relaxes a loader implementation detail that +applications could observe through cross-thread @code{dlopen} ordering. + +Setting this tunable to @samp{1} reverts to the pre-BZ-15686 behaviour: +@code{dl_load_lock} is held across constructor execution, giving a +strict total order of @code{dlopen} calls across threads at the cost of +reintroducing the BZ 15686 deadlock. This is intended as an escape +hatch for applications that implicitly relied on the old total order +(e.g., plugin registries whose registration order determined behaviour) +and need time to fix the underlying assumption. + +@strong{NB:} with @samp{1}, any @code{dlopen} of a DSO whose +constructor spawns a thread touching @code{thread_local} state (directly +or via libraries like glycin, gdk-pixbuf-glycin, NSS, etc.) will +deadlock. Use only as a temporary diagnostic aid. + +This tunable is @emph{temporary} and is scheduled for removal. As +noted above, tunables are not part of the @glibcadj{} stable ABI, and +this one is more constrained still: it exists only so that downstream +distributions and application authors can surface ordering regressions +introduced by the BZ 15686 fix while they address the underlying +assumptions. Because @theglibc{} reaches end users through downstream +distributions on the order of one to two years, the tunable is +expected to remain available for approximately four releases. It will +be removed once the ecosystem is shown to work with the new default. +Distributors and application authors should treat @samp{1} as a +stop-gap, not as a supported configuration: the correct long-term fix +is to remove implicit cross-thread @code{dlopen} ordering assumptions +from the affected code. +@end deftp + @node POSIX Thread Tunables @section POSIX Thread Tunables @cindex pthread mutex tunables diff --git a/sysdeps/pthread/Makefile b/sysdeps/pthread/Makefile index d0f3cd59ac..09b46e6d1c 100644 --- a/sysdeps/pthread/Makefile +++ b/sysdeps/pthread/Makefile @@ -349,6 +349,11 @@ tests += \ tst-atfork3 \ tst-atfork4 \ tst-create1 \ + tst-create2 \ + tst-create3 \ + tst-create4 \ + tst-create5 \ + tst-create6 \ tst-fini1 \ tst-pt-tls4 \ # tests @@ -365,6 +370,11 @@ modules-names += \ tst-atfork3mod \ tst-atfork4mod \ tst-create1mod \ + tst-create2mod \ + tst-create3mod \ + tst-create4mod-a \ + tst-create4mod-b \ + tst-create6mod \ tst-fini1mod \ tst-stack2-mod \ tst-tls4moda \ @@ -377,6 +387,9 @@ tst-atfork2mod.so-no-z-defs = yes tst-atfork3mod.so-no-z-defs = yes tst-atfork4mod.so-no-z-defs = yes tst-create1mod.so-no-z-defs = yes +tst-create2mod.so-no-z-defs = yes +tst-create4mod-a.so-no-z-defs = yes +tst-create4mod-b.so-no-z-defs = yes ifeq ($(build-shared),yes) # Build all the modules even when not actually running test programs. @@ -549,3 +562,35 @@ endif tst-stack2-TUNABLES += glibc.rtld.execstack=2 endif + +$(objpfx)tst-create2: $(shared-thread-library) +$(objpfx)tst-create2mod.so: $(shared-thread-library) +$(objpfx)tst-create2.out: $(objpfx)tst-create2mod.so + +$(objpfx)tst-create3: $(shared-thread-library) +$(objpfx)tst-create3.out: $(objpfx)tst-create3mod.so + +# tst-create4 verifies that a long-running constructor does not block +# concurrent dlopen of an unrelated library. Thread A's constructor +# blocks on a barrier; thread B dlopen's a different library. Before +# the BZ 15686 fix, dl_load_lock was held across the entire dlopen, +# including the constructor, so thread B deadlocked. After the fix, +# dl_load_lock is released during constructors and thread B succeeds. +LDFLAGS-tst-create4 = -Wl,-export-dynamic +$(objpfx)tst-create4: $(shared-thread-library) +$(objpfx)tst-create4mod-a.so: $(shared-thread-library) +$(objpfx)tst-create4mod-b.so: $(shared-thread-library) +$(objpfx)tst-create4.out: $(objpfx)tst-create4mod-a.so $(objpfx)tst-create4mod-b.so + +$(objpfx)tst-create5: $(shared-thread-library) + +# tst-create6 covers the combination that tst-create2 and tst-create3 +# exercise separately: the module constructor spawns and joins a +# thread (thread creation takes dl_load_tls_lock in +# _dl_allocate_tls_init) while a second thread concurrently dlopens +# the same DSO and waits for the constructor in dl_open_worker_begin. +# If that wait holds dl_load_tls_lock, the constructor's +# pthread_create blocks on it and the test deadlocks. +$(objpfx)tst-create6: $(shared-thread-library) +$(objpfx)tst-create6mod.so: $(shared-thread-library) +$(objpfx)tst-create6.out: $(objpfx)tst-create6mod.so diff --git a/sysdeps/pthread/tst-create2.c b/sysdeps/pthread/tst-create2.c new file mode 100644 index 0000000000..a419ed6b68 --- /dev/null +++ b/sysdeps/pthread/tst-create2.c @@ -0,0 +1,67 @@ +/* Verify that a thread spawned by a dlopen constructor can register a + TLS destructor via __cxa_thread_atexit_impl without deadlocking on + dl_load_lock held by the dlopen caller (BZ 15686). + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +/* Reproducer for one instance of the deadlock class described in + BZ 15686. + + thread 1: dlopen -> ctor -> pthread_create(worker) -> pthread_join(worker) + thread 2 (worker): __cxa_thread_atexit_impl -> tries to lock dl_load_lock + + Before the fix in elf/dl-open.c, dl_load_lock is held across + call_dl_init, so thread 2 blocks on a lock that thread 1 will only + release after pthread_join returns -- a deadlock that the + test-driver timeout surfaces as a failure. After the fix, + dl_load_lock is released before constructors run and reacquired + afterwards, so thread 2 makes progress and the dlopen call returns. + + Beyond "did not deadlock", the test also verifies that the TLS + destructor actually ran (tst_create2mod_dtor_done is set by dtor) + and that dlclose unloaded the DSO: the destructor has already + executed, so no reference is left on the module's + l_tls_dtor_count and dlopen with RTLD_NOLOAD must return NULL. */ + +#include <stdio.h> +#include <support/check.h> +#include <support/xdlfcn.h> + +static int +do_test (void) +{ + printf ("main: dlopen tst-create2mod.so\n"); + void *h = xdlopen ("tst-create2mod.so", RTLD_NOW); + printf ("main: dlopen done\n"); + + /* The worker thread exited before the constructor's pthread_join + returned, so its TLS destructor has already run. */ + int *dtor_done = xdlsym (h, "tst_create2mod_dtor_done"); + TEST_COMPARE (*dtor_done, 1); + + xdlclose (h); + printf ("main: dlclose done\n"); + + /* The destructor already ran, so no reference is left on the + module's l_tls_dtor_count and dlclose must have unloaded it. */ + TEST_VERIFY (dlopen ("tst-create2mod.so", RTLD_NOW | RTLD_NOLOAD) + == NULL); + + return 0; +} + +#include <support/test-driver.c> diff --git a/sysdeps/pthread/tst-create2mod.c b/sysdeps/pthread/tst-create2mod.c new file mode 100644 index 0000000000..92c475e439 --- /dev/null +++ b/sysdeps/pthread/tst-create2mod.c @@ -0,0 +1,74 @@ +/* Verify that a thread spawned by a dlopen constructor can register a + TLS destructor via __cxa_thread_atexit_impl without deadlocking on + dl_load_lock held by the dlopen caller (BZ 15686). + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <pthread.h> +#include <stdlib.h> +#include <dso_handle.h> + +extern int __cxa_thread_atexit_impl (void (*) (void *), void *, void *); + +/* Set to 1 by the TLS destructor when it runs. Read by the main + executable after dlopen to confirm the destructor executed. */ +int tst_create2mod_dtor_done; + +static void +dtor (void *obj) +{ + *(int *) obj = 1; +} + +/* The module TLS object mirrors the real-world trigger (a C++ + thread_local or Rust thread_local! first access), exercising + __tls_get_addr from the spawned thread as well. */ +static __thread int tls_obj; + +static void * +worker (void *arg) +{ + (void) arg; + + /* First touch of tls_obj forces __tls_get_addr, which on the + pre-BZ-15686 path is another dl_load_lock contender in addition + to __cxa_thread_atexit_impl below. */ + tls_obj = 1; + + /* Register the TLS destructor. Under the pre-fix locking model + __cxa_thread_atexit_impl acquires dl_load_lock, which is held by + the dlopen caller running this ctor, so the worker blocks here + and pthread_join in the constructor never returns. */ + if (__cxa_thread_atexit_impl (dtor, &tst_create2mod_dtor_done, + __dso_handle) != 0) + abort (); + + return &tls_obj; +} + +static void __attribute__ ((constructor)) +do_init (void) +{ + pthread_t t; + if (pthread_create (&t, NULL, worker, NULL) != 0) + abort (); + /* Blocks until worker has completed its __cxa_thread_atexit_impl + call; under the pre-fix locking model that call deadlocks on + dl_load_lock held by the dlopen caller running this ctor. */ + if (pthread_join (t, NULL) != 0) + abort (); +} diff --git a/sysdeps/pthread/tst-create3.c b/sysdeps/pthread/tst-create3.c new file mode 100644 index 0000000000..59b25908f3 --- /dev/null +++ b/sysdeps/pthread/tst-create3.c @@ -0,0 +1,112 @@ +/* Verify that concurrent dlopen of the same DSO is safe under the + per-DSO init serialisation added for BZ 15686. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +/* Two invariants are checked for concurrent dlopen of the same DSO: + + 1. The DSO's constructor runs EXACTLY ONCE, even if N threads race + into call_init. This catches a world where two threads both + think they lost the l_init_once CAS but actually proceed. + + 2. No dlopen caller returns before the constructor has finished. + A caller that beats the constructor would observe globals in + their BSS-zeroed state -- a regression from the pre-BZ-15686 + model where dl_load_lock serialised the whole _dl_open. + + The DSO constructor sleeps ~200 ms to widen the race window, then + publishes a "done" magic as its final write. Each thread checks + the magic immediately after dlopen returns; any thread observing + the wrong value has raced ahead of the constructor. + + The ctor-runs-once count is read from a handle that is kept open + across the check: every caller leaves its dlopen handle pinned in + the handles[] array and main reads tst_create3mod_ctor_count via + xdlsym on handles[0] before any dlclose. Closing the handles + first would unload the DSO once the last reference dropped, and a + subsequent re-open would load a fresh instance whose counter + starts at zero -- the count would then reflect only the + single-threaded re-open and could not detect a constructor that + ran more than once during the concurrent phase. */ + +#include <pthread.h> +#include <stdatomic.h> +#include <stdint.h> +#include <support/check.h> +#include <support/xdlfcn.h> +#include <support/xthread.h> + +#include "tst-create3.h" + +/* More threads than two so the race is exercised on multiple + pair-wise combinations; small enough to keep the test cheap. */ +#define NTHREADS 8 + +static pthread_barrier_t g_start_barrier; +static void *handles[NTHREADS]; + +static void * +worker (void *arg) +{ + int idx = (int) (intptr_t) arg; + + /* Release all workers at once so they enter _dl_open near-simultaneously. */ + xpthread_barrier_wait (&g_start_barrier); + + void *h = xdlopen ("tst-create3mod.so", RTLD_NOW); + + /* The "done" flag is the constructor's final write. If dlopen + returned before the constructor finished (the BZ 15686 race the + l_init_owner check in call_init prevents), this load will observe + 0 instead of TST_CREATE3_MAGIC_DONE. */ + _Atomic unsigned int *done = xdlsym (h, "tst_create3mod_done"); + unsigned int done_val = atomic_load_explicit (done, memory_order_acquire); + if (done_val != TST_CREATE3_MAGIC_DONE) + FAIL ("thread %d returned from dlopen before the constructor finished" + " (tst_create3mod_done=0x%x)", idx, done_val); + + /* Keep the handle open; main reads the ctor counter from it and + dlcloses all handles afterwards. */ + handles[idx] = h; + return NULL; +} + +static int +do_test (void) +{ + pthread_t threads[NTHREADS]; + + xpthread_barrier_init (&g_start_barrier, NULL, NTHREADS); + + for (int i = 0; i < NTHREADS; ++i) + threads[i] = xpthread_create (0, worker, (void *) (intptr_t) i); + + for (int i = 0; i < NTHREADS; ++i) + xpthread_join (threads[i]); + + /* The DSO is still loaded (all handles open), so the counter + reflects every constructor execution during the race above. */ + _Atomic int *count = xdlsym (handles[0], "tst_create3mod_ctor_count"); + TEST_COMPARE (atomic_load_explicit (count, memory_order_acquire), 1); + + for (int i = 0; i < NTHREADS; ++i) + xdlclose (handles[i]); + + return 0; +} + +#include <support/test-driver.c> diff --git a/sysdeps/pthread/tst-create3.h b/sysdeps/pthread/tst-create3.h new file mode 100644 index 0000000000..9a448ba2bf --- /dev/null +++ b/sysdeps/pthread/tst-create3.h @@ -0,0 +1,27 @@ +/* Shared definitions for tst-create3 and tst-create3mod. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#ifndef _TST_CREATE3_H +#define _TST_CREATE3_H + +/* Magic value published by the module constructor as its final action. + Any dlopen caller that observes tst_create3mod_done with a different + value immediately after dlopen returned has beaten the constructor. */ +#define TST_CREATE3_MAGIC_DONE 0xCAFEBABEu + +#endif diff --git a/sysdeps/pthread/tst-create3mod.c b/sysdeps/pthread/tst-create3mod.c new file mode 100644 index 0000000000..e21d473faf --- /dev/null +++ b/sysdeps/pthread/tst-create3mod.c @@ -0,0 +1,68 @@ +/* DSO for tst-create3: concurrent dlopen constructor-once test (BZ 15686). + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <stdatomic.h> +#include <time.h> + +#include "tst-create3.h" + +/* How long (nanoseconds) the constructor sleeps to simulate slow + initialisation. Large enough that, under the pre-fix locking model, + concurrent dlopen callers are very likely to reach call_init while + the constructor is still running. */ +#define TST_CREATE3_CTOR_SLEEP_NS 200000000 /* 200 ms */ + +/* Counter incremented by the constructor. Must be exactly 1 after + concurrent dlopen - catches the case where two threads both win the + CAS and run the constructor in parallel. */ +_Atomic int tst_create3mod_ctor_count = 0; + +/* Visibility marker. Set as the final write of the constructor with + release ordering. Concurrent dlopen callers must observe + TST_CREATE3_MAGIC_DONE here after their dlopen returns. */ +_Atomic unsigned int tst_create3mod_done = 0; + +static void +sleep_ns (long ns) +{ + struct timespec ts = + { + .tv_sec = ns / 1000000000L, + .tv_nsec = ns % 1000000000L + }; + nanosleep (&ts, NULL); +} + +static void __attribute__ ((constructor)) +do_init (void) +{ + /* Record that we ran. Two threads winning the l_init_once CAS + would increment this more than once. */ + atomic_fetch_add_explicit (&tst_create3mod_ctor_count, 1, + memory_order_relaxed); + + /* Slow the constructor down to widen the window in which a buggy + call_init would let a concurrent caller return from dlopen. */ + sleep_ns (TST_CREATE3_CTOR_SLEEP_NS); + + /* Publish "constructor finished" as the last write with release + ordering, so that callers observing TST_CREATE3_MAGIC_DONE also + observe every earlier write the constructor made. */ + atomic_store_explicit (&tst_create3mod_done, TST_CREATE3_MAGIC_DONE, + memory_order_release); +} diff --git a/sysdeps/pthread/tst-create4.c b/sysdeps/pthread/tst-create4.c new file mode 100644 index 0000000000..8c7eeae517 --- /dev/null +++ b/sysdeps/pthread/tst-create4.c @@ -0,0 +1,100 @@ +/* Verify that a long-running dlopen constructor does not block a + concurrent dlopen of an unrelated library. + + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +/* WHAT THIS TEST CHECKS + + Thread A calls dlopen on a library whose constructor blocks on a + barrier. Once the constructor is executing (i.e. dl_load_lock has + been released by the BZ 15686 fix), thread B calls dlopen on an + unrelated library. + + Before the BZ 15686 fix, dl_load_lock was held across the entire + dlopen call, including constructor execution, so thread B's dlopen + would block on dl_load_lock forever. After the fix, dl_load_lock + is released for the duration of the constructor, so thread B's + dlopen proceeds in parallel. + + The test is deterministic (no probabilistic race): we spin until + the constructor confirms it has started, then launch thread B. */ + +#include <pthread.h> +#include <stdatomic.h> +#include <sched.h> +#include <stdio.h> +#include <support/xdlfcn.h> +#include <support/xthread.h> + +#include "tst-create4.h" + +/* Exported for the DSOs via -Wl,-export-dynamic (LDFLAGS-tst-create4). */ +pthread_barrier_t tst_create4_ctor_barrier; +atomic_int tst_create4_ctor_running = 0; + +static void * +worker_a (void *unused) +{ + (void) unused; + void *h = xdlopen ("tst-create4mod-a.so", RTLD_NOW); + xdlclose (h); + return NULL; +} + +static void * +worker_b (void *unused) +{ + (void) unused; + void *h = xdlopen ("tst-create4mod-b.so", RTLD_NOW); + xdlclose (h); + return NULL; +} + +static int +do_test (void) +{ + xpthread_barrier_init (&tst_create4_ctor_barrier, NULL, 2); + + /* Start thread A. It enters the constructor of tst-create4mod-a, + which sets ctor_running = 1 and then blocks on the barrier. */ + pthread_t ta = xpthread_create (0, worker_a, NULL); + + /* Spin until the constructor has definitely started. */ + while (atomic_load_explicit (&tst_create4_ctor_running, + memory_order_acquire) == 0) + sched_yield (); + + /* Now launch thread B. If dl_load_lock is released during + constructors (BZ 15686 fix), thread B's dlopen will succeed. + If the lock is still held by thread A, thread B blocks on + dl_load_lock and the test will time out. */ + pthread_t tb = xpthread_create (0, worker_b, NULL); + + /* Wait for thread B to finish. */ + xpthread_join (tb); + + printf ("info: concurrent dlopen of unrelated library succeeded\n"); + + /* Signal the barrier so thread A's constructor can return. */ + xpthread_barrier_wait (&tst_create4_ctor_barrier); + xpthread_join (ta); + + return 0; +} + +#include <support/test-driver.c> diff --git a/sysdeps/pthread/tst-create4.h b/sysdeps/pthread/tst-create4.h new file mode 100644 index 0000000000..2e6b79a702 --- /dev/null +++ b/sysdeps/pthread/tst-create4.h @@ -0,0 +1,37 @@ +/* Shared definitions for tst-create4 and its modules. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#ifndef _TST_CREATE4_H +#define _TST_CREATE4_H + +#include <pthread.h> +#include <stdatomic.h> + +/* Barrier that the module constructor waits on. Main signals it after + the concurrent dlopen of the unrelated module completes. Defined in + the main executable and exported to the modules via the dynamic symbol + table (-Wl,-export-dynamic). */ +extern pthread_barrier_t tst_create4_ctor_barrier; + +/* Flag set to 1 by the module constructor once it has started (i.e. + after dl_load_lock has been released by the BZ 15686 fix). Main + spins on this flag to guarantee the constructor is executing before + it creates the second worker thread. */ +extern atomic_int tst_create4_ctor_running; + +#endif diff --git a/sysdeps/pthread/tst-create4mod-a.c b/sysdeps/pthread/tst-create4mod-a.c new file mode 100644 index 0000000000..d5a44e431a --- /dev/null +++ b/sysdeps/pthread/tst-create4mod-a.c @@ -0,0 +1,30 @@ +/* DSO A for tst-create4: constructor blocks on a barrier, simulating a + long-running initializer (e.g. glycin spawning a sandbox process). + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <stdatomic.h> + +#include "tst-create4.h" + +static void __attribute__ ((constructor)) +init_a (void) +{ + atomic_store_explicit (&tst_create4_ctor_running, 1, + memory_order_release); + pthread_barrier_wait (&tst_create4_ctor_barrier); +} diff --git a/sysdeps/pthread/tst-create4mod-b.c b/sysdeps/pthread/tst-create4mod-b.c new file mode 100644 index 0000000000..d65340b971 --- /dev/null +++ b/sysdeps/pthread/tst-create4mod-b.c @@ -0,0 +1,22 @@ +/* DSO B for tst-create4: trivial constructor that just returns. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +static void __attribute__ ((constructor)) +init_b (void) +{ +} diff --git a/sysdeps/pthread/tst-create5.c b/sysdeps/pthread/tst-create5.c new file mode 100644 index 0000000000..34edf17dac --- /dev/null +++ b/sysdeps/pthread/tst-create5.c @@ -0,0 +1,63 @@ +/* Verify that dlopen(NULL) from a worker thread does not deadlock + after the main executable has been initialized. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +/* Reproducer for a regression in the BZ 15686 fix. + + The main executable's constructors are not run by call_init in + elf/dl-init.c; instead they are handled by the startup code. With + the per-DSO init serialization added for BZ 15686, call_init must + nevertheless mark the main executable as fully initialized by + setting l_init_once = 2. Otherwise a later multi-threaded + dlopen(NULL) / __RTLD_OPENEXEC takes the already-loaded early-return + path in dl_open_worker_begin, sees l_init_called == 1 but + l_init_once != 2, and waits forever for a constructor that will + never complete. + + This test spawns a worker thread that calls dlopen(NULL, RTLD_NOW). + If the bug is present, the call deadlocks and the test-driver + timeout surfaces the failure. After the fix, dlopen(NULL) returns + immediately. */ + +#include <stdio.h> +#include <support/xdlfcn.h> +#include <support/xthread.h> + +static void * +worker (void *arg) +{ + (void) arg; + + dprintf (1, "worker: dlopen(NULL)\n"); + void *h = xdlopen (NULL, RTLD_NOW); + dprintf (1, "worker: dlopen(NULL) done\n"); + xdlclose (h); + return NULL; +} + +static int +do_test (void) +{ + pthread_t t = xpthread_create (0, worker, NULL); + xpthread_join (t); + + dprintf (1, "main: worker finished\n"); + return 0; +} + +#include <support/test-driver.c> diff --git a/sysdeps/pthread/tst-create6.c b/sysdeps/pthread/tst-create6.c new file mode 100644 index 0000000000..3575226860 --- /dev/null +++ b/sysdeps/pthread/tst-create6.c @@ -0,0 +1,111 @@ +/* Verify that a concurrent dlopen caller waiting for another thread's + in-progress constructor does not hold dl_load_tls_lock while it + waits (BZ 15686 follow-up). + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +/* WHAT THIS TEST CHECKS + + The BZ 15686 constructor fix releases dl_load_lock around + call_dl_init and makes a concurrent dlopen of the same DSO wait in + dl_open_worker_begin until the constructor completes. That wait + runs while dl_open_worker holds dl_load_tls_lock. If the + constructor spawns a thread, thread creation needs dl_load_tls_lock + (_dl_allocate_tls_init), producing a deadlock cycle: + + waiter: holds dl_load_tls_lock, waits for ctor end (futex) + loader: runs ctor, waits for pthread_create -> dl_load_tls_lock + + This test combines the two ingredients that tst-create2 and + tst-create3 exercise separately: a constructor that spawns and + joins a thread (tst-create6mod.c), and two threads concurrently + dlopening the same DSO. The constructor sleeps 200 ms before + pthread_create so the second caller reliably reaches the + early-return wait first. + + Without the dl_load_tls_lock release around the wait, the test + deadlocks and the test-driver timeout fires; with it, both dlopen + calls return only after the constructor published its done magic, + and the constructor ran exactly once. */ + +#include <pthread.h> +#include <stdatomic.h> +#include <stdint.h> +#include <support/check.h> +#include <support/xdlfcn.h> +#include <support/xthread.h> + +#include "tst-create6.h" + +/* Two threads: one becomes the loader running the constructor, the + other becomes the waiter. More waiters would not change the + mechanism. */ +#define NTHREADS 2 + +static pthread_barrier_t g_start_barrier; +static void *handles[NTHREADS]; + +static void * +worker (void *arg) +{ + int idx = (int) (intptr_t) arg; + + /* Release both workers at once so they enter _dl_open + near-simultaneously. */ + xpthread_barrier_wait (&g_start_barrier); + + void *h = xdlopen ("tst-create6mod.so", RTLD_NOW); + + /* If dlopen returned before the constructor finished, the done + magic is not yet visible. */ + _Atomic unsigned int *done = xdlsym (h, "tst_create6mod_done"); + unsigned int done_val = atomic_load_explicit (done, memory_order_acquire); + if (done_val != TST_CREATE6_MAGIC_DONE) + FAIL ("thread %d returned from dlopen before the constructor finished" + " (tst_create6mod_done=0x%x)", idx, done_val); + + /* Keep the handle open so the ctor counter below reflects the + concurrent phase. */ + handles[idx] = h; + return NULL; +} + +static int +do_test (void) +{ + pthread_t threads[NTHREADS]; + + xpthread_barrier_init (&g_start_barrier, NULL, NTHREADS); + + for (int i = 0; i < NTHREADS; ++i) + threads[i] = xpthread_create (0, worker, (void *) (intptr_t) i); + + for (int i = 0; i < NTHREADS; ++i) + xpthread_join (threads[i]); + + /* Both dlopen calls returned; the DSO is still loaded via the + pinned handles. The constructor must have run exactly once. */ + _Atomic int *count = xdlsym (handles[0], "tst_create6mod_ctor_count"); + TEST_COMPARE (atomic_load_explicit (count, memory_order_acquire), 1); + + for (int i = 0; i < NTHREADS; ++i) + xdlclose (handles[i]); + + return 0; +} + +#include <support/test-driver.c> diff --git a/sysdeps/pthread/tst-create6.h b/sysdeps/pthread/tst-create6.h new file mode 100644 index 0000000000..7179dfdd8d --- /dev/null +++ b/sysdeps/pthread/tst-create6.h @@ -0,0 +1,26 @@ +/* Shared constants for tst-create6 and tst-create6mod. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#ifndef TST_CREATE6_H +#define TST_CREATE6_H + +/* Final value published by the module constructor as its last write, + after the spawned thread has been created and joined. */ +#define TST_CREATE6_MAGIC_DONE 0x5eed5eedu + +#endif /* TST_CREATE6_H */ diff --git a/sysdeps/pthread/tst-create6mod.c b/sysdeps/pthread/tst-create6mod.c new file mode 100644 index 0000000000..d4aac319ba --- /dev/null +++ b/sysdeps/pthread/tst-create6mod.c @@ -0,0 +1,72 @@ +/* DSO for tst-create6: its constructor spawns and joins a thread. + Thread creation takes dl_load_tls_lock in _dl_allocate_tls_init, + so this exercises the BZ 15686 follow-up fix: a concurrent dlopen + caller waiting for this constructor in dl_open_worker_begin must + not hold dl_load_tls_lock while it waits. + Copyright (C) 2026 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + <https://www.gnu.org/licenses/>. */ + +#include <pthread.h> +#include <stdatomic.h> +#include <stdlib.h> +#include <time.h> + +#include "tst-create6.h" + +/* Number of times the constructor ran. Must end up == 1. */ +_Atomic int tst_create6mod_ctor_count = 0; + +/* Published as the constructor's final write; dlopen callers check + it to prove they did not return before the constructor finished. */ +_Atomic unsigned int tst_create6mod_done = 0; + +static void * +worker (void *arg) +{ + (void) arg; + /* Merely existing is enough: creating this thread already required + dl_load_tls_lock. */ + return NULL; +} + +static void __attribute__ ((constructor)) +do_init (void) +{ + atomic_fetch_add_explicit (&tst_create6mod_ctor_count, 1, + memory_order_relaxed); + + /* Give a concurrent dlopen caller time to reach the early-return + wait in dl_open_worker_begin before we need dl_load_tls_lock + below. */ + struct timespec ts = { .tv_nsec = 200000000 }; /* 200 ms */ + nanosleep (&ts, NULL); + + /* pthread_create -> allocatestack -> _dl_allocate_tls_init takes + dl_load_tls_lock. If the concurrent dlopen caller is blocked in + dl_open_worker_begin while holding dl_load_tls_lock (the bug this + test regresses), this call never returns and pthread_join below + never completes: the test hangs and the test-driver timeout + fires. */ + pthread_t t; + if (pthread_create (&t, NULL, worker, NULL) != 0) + abort (); + if (pthread_join (t, NULL) != 0) + abort (); + + atomic_store_explicit (&tst_create6mod_done, TST_CREATE6_MAGIC_DONE, + memory_order_release); +} -- 2.51.0