Re: Maxima and thread safety

*Regarding the TLS size:*
I agree that for most realistic applications, setting the TLS limit to a
high enough value acts as a band-aid. We have already increased it
significantly compared to the default, yet Maxima still crashes after
running rtest_rules just a few times (which doesn't actually create
terribly many rules). Knowing that these TLS slots will never be freed and
just keep filling up feels like a ticking time bomb.

*Regarding special variables and TLS:*
A special variable by itself doesn't inherently live in TLS. It is only
when you dynamically bind it (e.g., via let or prog) that the thread gets
its own private copy in TLS. If it is not dynamically bound, the thread
falls back to accessing the global value cell, which is shared across all
threads. This is exactly where synchronization mechanisms like mutexes
become necessary to prevent chaos - but this must be done by the developer.
Variables like $values or $functions fall into this category: they aren't
locally bound during execution, so any concurrent mutation to them affects
the shared global state.
Furthermore, standard Lisp list operations like push, rplaca, or rplacd are
not atomic. Two threads can concurrently try to mutate the exact same list
structure, inevitably leading to lost updates or tangled pointers.
Compounding this is the complex hardware issue of memory visibility -
without explicit synchronization (like memory barriers), there is no
guarantee when, or even if, one thread will actually "see" the memory
changes just made by another. The absence of any such guarantee is what
allows modern CPUs with their many cores and multi-layered caches to be so
fast.

*Regarding SBCL's source code:*
The reason you don't see mutexes in standard Lisp operations is that Common
Lisp data structures (like lists and hash tables) make zero guarantees
about thread safety (as far as I know). If they did, every basic operation
would be drastically slower due to synchronization overhead. If a developer
wants thread safety for their data, they have to implement it themselves.

Multi-threading support is a fundamental architectural design decision that
is difficult to retrofit - especially for a program like Maxima, which was
written decades before multi-threading was even a concept.

Best regards
David Scherfgen


Michel Talon <[email protected]> schrieb am So., 7. Juni
2026, 21:38:

>
> Le 07/06/2026 à 16:55, David Scherfgen a écrit :
>
> Following up on the discussion about Maxima's thread safety ...
>
> I've been playing around a bit with Michel's
> distribute_over_tranches_thread.lisp
> <https://github.com/maxima-project-on-github/maxima-packages/blob/master/mtalon/maxima-parallel/distribute_over_tranches_thread.lisp>.
> It works nicely when the evaluated expressions are free of side effects
> ("pure functions").
>
> Unfortunately, Maxima has many hidden side effects that are surprising to
> the average user. The following innocent looking example simply creates
> Maxima variables v1, v2, v3, ..., v10000 in 16 parallel threads:
>
> distribute_over_tranches('(concat('v, i) :: i), i, 10000, 16)$
>
> Note: The double-colon operator :: is like :, but it evaluates its
> left-hand side.
>
> Now that looks completely safe, right? After all, each parallel evaluation
> creates a different variable. But under the hood, new variables are added
> to $values, and being a true global variable, this is not thread-safe.
> Different threads manipulate the $values list in parallel, getting in
> each other's way. After executing the above line, let's see how many
> variables Maxima thinks exist:
>
> length($values);
> 9544 /* oops, not 10000! (non-deterministic) */
>
> If we start evaluating limits in parallel, it breaks Maxima completely:
>
> distribute_over_tranches('(limit(abs(x-i)/(x-i), x, i)), i, 100, 100);
>
> This prints lots of low-level errors, and afterwards, Maxima is in a
> corrupted state where even regular, non-parallelized limits don't work any
> more. This is probably caused by database facts being added by limit,
> which is also not thread-safe, but I'm not sure. Integrals are equally
> problematic.
>
> These experiments confirm that while parallel evaluation of pure functions
> is achievable, parallel mutation of the Maxima environment is currently
> impossible without deep architectural changes, which, given the complexity
> and age of the codebase, are highly challenging.
>
> Best regards
> David Scherfgen
>
> I agree with everything you say. However i think that the original problem
> (the question of rules, etc.) is a non problem because one can increase the
> size of thread local storage which is extremely small by default (so that
> one can spawn threads in the thousands (for example for a network server)
> but this is almost totally improbable for maxima. I cannot see how one can
> realistically hit the limit for any reasonable maxima program if one sets o
> good size for thread local storage.
>
> Also it was my understanding that all variables defined by defvar,
> defparameter or declared special are assigned to thread local storage, and
> that most maxima variables accessible to the end user are declared special.
> If not, how can they be "global" and in what sense? I agree with you that
> objects to which they refer are problematic, and require thinking about.
> Anyways i have been quite surprised to look at the source code of sbcl and
> see that it is usually very ordinary looking with no profusion of mutexes,
> etc. still it is supposed to be thread safe.
>
>
> --
> Michel Talon
>
>

_______________________________________________
Maxima-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/maxima-discuss