Re: Threads.... the long story, and a proposal

On Fri, Sep 4, 2026 at 11:29 AM Leon Timmermans <[email protected]> wrote:
>
[...]
>
> Sorry but I don't think people are going to read a 16k word proposal from a stranger.
>
> Maybe you have some good ideas here, but we're not going to find out if it's presented like this.
>
> Leon

Fair enough.

Let's start with stackfull green threads...

A stackful green thread library has to swap the interpreter's mutable
execution registers.
Core owns that set already, but there is no way to ask for it, so stackful
green threads implementations (Coro) keeps its own transcription,
with version guards to deal with the different Perl versions.
Perl core legitimately don't want to expose its internals as public API.
Coro author doesn't commit to support anymore its Core internal copies.
This lead to several breakage. Since, Perl 5.24, Coro is broken again.
I had to fix this on my side, and I searched how to better address this
problem (see also https://github.com/Perl/perl5/issues/18093 actually).

The proposal is to publish the set, generated from a single X-macro in
execstate.h shared with init_stacks, so that the API and the live
interpreter cannot drift apart.  This API should be usable by any stackful
green thread, not just Coro.

The operations are:
- save and load a register snapsho for the context switch, the caller doing
  its own C-stack switch in between,
- set up a fresh set of stacks with init_stacks' invariants, unwind
  one that will never run again, and free it,
- derive and free a padlist, for re-entering a sub on an independent
  context,
- the JMPENV transfer registers, as lvalue aliases and two helpers.

Cost to core: ~800 lines added, and nothing removed or altered.
No existing function signature, struct layout, macro or
code path changes.  PerlExecState is standalone storage: it is never
embedded in, aliased over, or used to redeclare any part of
PerlInterpreter.  With the API unused, the interpreter runs exactly as
it does today.

It is implemented against 5.42.2 with tests in XS-APItest (t/execstate.t),
and Coro has been ported to it: the register list drops from 52 entries to 9
(which are legitimately Coro's responsibility).
This is also tested in a real world application.

Out of scope:  the C-stack switch (libcoro, the assembler
backends) stays in Coro.

The technique, the register list, the fresh-stack setup and the unwind
ordering are Marc Lehmann's work in Coro. LLM used for the split,
tests, and documentation (and validation in different contexts, including
Linux and Windows).
Coro is licensed on the same terms as perl, so there is no licensing
obstacle.

I wonder whether an additive API of this size and shape is
acceptable in principle.  If it is, I will submit the patch for review.
The API definition on its own is in Porting/execstate_api.md; the longer
rationale, the alternatives considered and the comparison with the
stackless model (Future::AsyncAwait) are in
Porting/green_threads_and_execstate.md (the study), for anyone who
wants them.


Side note:
The rest of the study is really to understand, as far as possible, all the
aspects of threading in Perl (and sorry for the size of the doc, but even in
the LLM boring style, it is quite instructive, and required quite some human
work behind as well, as a lot of this derives from my experience and
research).
This is to echo the last thread on this 5 years ago ("threading and stuff"),
which asked for quite a more extensive study. Which now exists (and I'd be
happy to have feedback on it).

IMO, one needs to get all the information why Coro (or rather stackful green
threads) would still be an asset to Perl, as otherwise this proposal will go
nowhere.
Also, it does not just cover Coro, but FAA as well, as I seriously considered
it (and reported several bugs and fixes on it) before deciding to put some
efforts into this here (which is why there are quite some work on this side as
well, even if no more my primary target).

Thanks,

-- Laurent