Re: [PATCH v3] elf: Release dl_load_lock before running dlopen constructors (BZ 15686)

On 7/20/26 1:12 PM, Alexander Pevzner wrote:
> On 7/19/26 4:52 PM, Adhemerval Zanella Netto wrote:
>> From the the reported deadlock, it doesn't really require
>> releasing dl_load_lock around constructors at all. The actual
>> blocking site is __cxa_thread_atexit_impl, which takes
>> dl_load_lock only to look up the caller's link map. Replacing that
>> lookup with the lock-free _dl_find_object removes the lock
>> acquisition entirely.
> 
> The issue is actually more fundamental than that.
> 
> Because the current implementation holds a global lock during the
> entire dlopen() operation, concurrent dlopen() calls are susceptible
> to deadlocks. Specifically, if a constructor in the first thread
> tries to synchronize or communicate with a second thread that is
> currently blocked waiting for the dlopen() lock, a deadlock occurs.
> 
> While this application architecture is unusual, it is not prohibited
> by POSIX or glibc documentation.
> 
> Replacing the lock in __cxa_thread_atexit_impl resolves this
> specific instance, but it leaves the root architectural problem
> unfixed.
(1) Solving the narrower problem as a short-term solution.

Adhemerval's solution seems like a good immediate fix for the problem.

Does it fix the issue?

As a project lead I would like to see Bug 15686 fixed, but if it takes
us time to get to that solution, we also want to resolve the problem
that you're seeing in a timely fashion and are willing to accept a short-term
solutions to help you out.

(2) Solving the architectural problem may take longer or require more testing.

Large changes like those which are being proposed here will take time to review
and check against existing application expectations.

The implementation is constrained by more than just what the standards
say, or the glibc manual says, it is constrained by expectations that
applications may have created.

It should not be this way, but it is, and we attempt to consider such
constraints when making changes because we don't want to knowingly break
such applications unless there is a meaningful and valuable gain.

We might be able to run with this change downstream in Fedora and use
our build infrastructure to do A/B build testing of the distribution to see if
something comes out of the larger scale testing.

I'm excited to see the results of ROSA Labs running with these changes in
production and reporting back the results.

(3) POSIX and glibc say nothing about this today --- making this a hard problem.

The problem *is* that we say nothing about what is and is not allowed and have
permitted the implementation to grow to the point that users expect a completely
flexible asynchronous processing of dependencies across multiple threads
allowing calls to all library functionality.

For example, some users expect a malloc interposer can call dlopen, but this
clearly has to have some constraints. If dlopen needs malloc then it doesn't
work as the solution has a dependency cycle (discussed previously on list as a
new form of safety e.g. "synchronous reetrancy").

Since we have never said anything before about this, as Adhemerval notes, this
area is going to possibly cause problems with existing applications, and for
which this patch provides a tunable.

It would be better if instead of using a tunable we spent more time evaluating
the problem space and a solution, hence I think Adhemerval's suggestion to use
an intermediate solution. This intermediate solution is not a final solution,
but an option to resolve the short-term problem while we take more time to
evaluate a full solution.

In truth we say a few things about this in the glibc manual in the Dynamic Linker
chapter, but only to warn against having any expectations :-)

(4) A full solution requires understanding what we're asking --- and possibly tooling.

Are we asking that all foreign function callbacks can call into the library safely?

Are we only asking that all foreign function callbacks for library initializers and
finalizers be able to call back into the library safely?

Constructors and destructors are foreign function calls made during library
initialization and finalization, and as such we have never discussed what is and
isn't permitted to be called in those functions.

These foreign functions are not occurring in the normal context of the application
runtime, nor do we provide ways for them to know what *other* non-libc functionality
can be safely accessed and if it will ever change (based on load order).

Is there any way we can add tooling or _dl_debug_printf options to detect problems?

Example: lari: https://docs.oracle.com/cd/E88353_01/html/E37839/lari-1.html

Should we add the option to randomly order dependencies whose relative order shouldn't
matter? To shake out problems in development?

I think a solution to this problem is going to take time, particularly to evaluate
the effect on existing applications and runtimes.

Thoughts on (1), (2), (3) or (4)?

-- 
Cheers,
Carlos.