Re: SBCL Fibers proposal

On Sun, Mar 1, 2026 at 10:18 AM Christophe Rhodes <[email protected]> wrote:

> Anthony Green <[email protected]> writes:
>
> > My fibers patches aren't ready to be submitted yet, but I've posted
> > the proposal, including many details and some benchmark results here:
> > https://atgreen.github.io/repl-yell/posts/sbcl-fibers/
>
> I think it would be helpful for readers -- at least me -- to know which
> parts of this proposal you, personally, have reviewed in detail and are
> confident in the exact wording published; which parts have been
> generated by some process and look generally plausible to you; and which
> parts have not been reviewed by you at all.
>

This is a very fair question.   I worked with the LLM to generate the
proposal based on the current implementation and I am confident in the
exact wording.  I can explain every part of the design in detail because I
guided the LLM based on experience and knowledge of other implementations.
However, I admit I haven't reviewed every line of code my junior developer
wrote yet.   This is one of the gates I must pass before submitting the
patches.  They need to be reorganized and cleaned up into a series of
digestible changes before I do this.


> I also think it would be helpful for readers to understand which parts
> of the design and implementation are fixed in stone, and which parts are
> design choices (and for those choices, which other choices were
> considered).
>

Nothing is set in stone, but I feel like this is pretty close to the best
co-routines design you can get as a purely additive feature.  The
implementation was heavily influenced by goroutines and java virtual
threads.

  API: fixed

  - Cooperative yield model (fiber-yield, fiber-park, fiber-sleep). This is
the core contract. Fibers yield explicitly at known points. Changing to
preemptive would be a different system entirely.
  - make-fiber + submit-fiber as separate create/start steps. This mirrors
thread APIs and allows pre-configuration before scheduling.
  - fiber-join returning result + errorp. This is the standard
join-with-result pattern (similar to ignore-errors -- thanks phoe!)
  - Fiber-aware overloads of mutex, condition-wait, wait-until-fd-usable,
sleep. These have to exist for fibers to be useful with existing SBCL code.
The alternative is making users rewrite all blocking calls, which defeats
the purpose.
  - run-fibers as the simple batch API. Straightforward enough that there's
no reason to change it.

  API: design choices that could reasonably go differently

  - start-fibers / submit-fiber / finish-fibers as the dynamic API. I like
this, but a more explicit close-group or channel-based completion model
could replace it.
  - Fiber pinning (fiber-pin/fiber-unpin/with-fiber-pinned). The need is
real (FFI, thread-local resources), but the pin-count API is one option. An
alternative would be a with-fiber-affinity that binds to a specific
carrier, or maybe even automatic pinning when FFI calls are detected.

  Implementation: could change without breaking the API

  - Chase-Lev deque vs. global queue vs. some other scheduler structure.
Users don't see the deque.  I like the deque approach as avoids lock
contention.
  - epoll/kqueue/poll backend — this is already platform-abstracted behind
wait-until-fd-usable. Could switch to io_uring without API changes.
  - Stack pooling strategy (madvise vs. munmap/mmap vs. something else).
Users set :stack-size and don't care how stacks are recycled.
  - GC integration approach (two-list, thread struct slot, etc.). Invisible
to users as long as GC correctness is maintained.
  - Binary heap vs. timing wheel for deadlines. Users call fiber-sleep and
don't know the data structure.
  - Random victim selection for work stealing. Could change to
power-of-two-choices or affinity-aware stealing with no API impact.

AG

_______________________________________________
Sbcl-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sbcl-devel