Re: Capy Review, take 2

On 6/29/26 16:31, Vinnie Falco via Boost wrote:
> On Mon, Jun 29, 2026 at 3:25 AM Rainer Deyke via Boost <
> [email protected]> wrote:
>> IoAwaitable exists to support the guarantee that every task runs
>> on its executor. This guarantee is useful for making it easier to
>> reason about code and for writing correct code. It also comes at
>> a considerable runtime cost. Coroutines get bounced around between
>> IoAwaitables and executors like a game of ping-pong.
> 
> The "bouncing" framing doesn't match how the library is used in practice.
> In the primary use case, (networking) you launch a coroutine chain on an
> executor (a strand, an io_context) and the entire chain stays there.
> No executor
> hopping. The executor affinity is the point: you set it once and stop
> thinking about it.

You're still thinking of the library as primarily an i/o library, while 
I'm thinking of it as a general coroutine library that's going to 
subsume all coroutine use in the C++ world.  Can you really not imagine 
a use where executor jump overhead might be significant, or do you just 
not want to support that use?

My primary motivating example remains:

   task<> load_texture(std::string path) {
     // Step 1
     auto image_data = co_await load_file(path);

     // Step 2
     auto decoded_image = decode_webp(image_data);

     // Step 3
     capy::run(main_thread_executor)(
       [decoded_image = std::move(decoded_image)] {
       upload_image_to_opengl(decoded_image);
     }
   }

Step 1 doesn't really matter because it spends all of its time 
co_awating.  Step 2 is CPU-bound and needs to run in a thread pool. 
Step 3 must run on the main thread.

At least one executor jump is necessary, and Capy provides it, at the 
cost of a second, unnecessary jump.  And if I just call the coroutine 
once, there's no problem.  But the strength of coroutines over threads 
is that they have less overhead, so I can have more of them.  What if I 
want to load thousands of tiny textures?  What if I want to load millions?

(That's still an artificial example, because loading millions of 
textures as individual files is terrible idea for reasons that have 
nothing to do with the overhead of jumping between executors.)

But there's more.  Even without capy::run or resume_on, every single 
call to co_await potentially involves a post operation, i.e. an executor 
jump.  It's part of the IoAwaitable protocol.  Every IoAwaitable 
potentially suspends a coroutine (because otherwise it would be a plain 
function), and every suspended coroutine potentially posts (because 
that's what Executor::dispatch does when not already running on the 
correct executor).  How often this happens depends the IoAwaitable in 
question and the context it's used in.  Synchronization structures 
(async_mutex, async_event) require posts when they are used to 
synchronize between different executors.  capy::delay always requires a 
post.  Custom IoAwaitables that wrap third party asynchronous APIs may 
also require a post every time they are used.

Every time co_await is used, a coroutine may be suspended, only to be 
resumed later in response to some sort of trigger.  Sometimes the 
trigger comes from another coroutine running on the same executor. 
Sometimes the trigger comes from a coroutine running on a different 
executor.  Sometimes it comes from a worker thread that is not part of 
any executor, as in the capy::delay example.  Sometimes it comes from 
the main thread, responding to user input.  In exactly one of these 
cases can the IoAwaitable avoid posting the coroutine to its executor.

And even if the trigger comes from a coroutine on the same executor, 
posting may be necessary.  Consider capy::async_mutex::unlock: even when 
called from the same executor, there is no trampoline available for 
symmetric transfer because capy::async_mutex::unlock is a function, not 
a coroutine or an IoAwaitable.

Posts from co_await are so common that I find it best to assume that 
every co_await will involve at least one post operation by default, 
until proven otherwise.

(Maybe Corosio solves this problem for socket i/o, by providing 
IoAwaitables that always resume on the correct executor without the 
overhead of a post operation.  If so, that's a great achievement, but 
not one that solves the problem for other uses of coroutines.)

And to be perfectly clear: resume_on can do nothing about most of these 
posts, but immediate_executor can.  resume_on exists for ergonomics, 
with some potential performance gain as a side benefit. 
immediate_executor exists for performance, at a significant ergonomic cost.

> 1. the library provides the tools for anyone to implement `resume_on`
> themselves. The executor concept, `continuation`, `dispatch`, and `post`
> are all public. Nothing prevents a user from building `resume_on` as a
> standalone awaitable. We don't have to ship it to enable it.
> 
> 2. `resume_on` breaks the environment model. When you `resume_on(ex)` and
> then `co_await` an IoAwaitable, the `io_env` still points to the *original*
> executor. The IoAwaitable dispatches its completion to the wrong executor.
> To fix this, you either need to update the `io_env` (but it's `const*` and
> shared - who owns the new one?) or allocate a new `io_env` (which is
> exactly what `run` already does). So `resume_on` either breaks the
> environment or duplicates the machinery of `run`, minus the safety of
> scoped lifetime.

To me, those two points are contradictory, and explain why I think that 
Capy needs to make a stand one way or another.  There are two 
possibilities.  Either it's possible to write a "correct" resume_on that 
can be safely used with Capy, or it's not.

In other former case, the gotchas associated resume_on are precisely why 
it should *not* be left to the users to write.  Point 2 overrides point 
1.  resume_on is too dangerous to be left to users.

In the latter case, point 1 is invalid.  The user *cannot* write 
resume_on because resume_on is broken on a fundamental, conceptual 
level.  And this should be documented by Capy, to prevent the user from 
trying.


>> One caveat about resume_on: its effect should be limited to the
>> coroutine in which it is used. When coroutine A co_awaits coroutine
>> B, and coroutine uses resume_on and then co_returns, execution on
>> coroutine A should always resume on A's original executor, not the
>> executor that B switched to.
> 
> This caveat is exactly what makes `resume_on` as complex as `run`. When
> coroutine B calls `resume_on(ex2)` and then `co_return`s, coroutine A must
> resume on A's original executor. That means `final_suspend` must know which
> executor the parent was on and dispatch back to it. That is exactly what
> the trampoline in `run` does. The "simpler" `resume_on` requires the same
> machinery as `run` to be correct. The complexity is merely hidden not
> removed.

Simplicity of implementation was never the goal.  On the contrary, I 
assumed that capy::run exists because the more ergonomic resume_on 
interface proved too difficult to implement correctly.  I am still under 
that impression.

The strength of coroutines is that they allow chains of callbacks to be 
written as if they were linear functions.  Using capy::run reads like a 
callback.  Using resume_on reads like a linear function.

Callback version:

   task<> f() {
     auto image = co_await load_image();
     main_thread_callbacks.push_back([image = std::move(image)] {
       upload_texture_to_opengl(image);
     });
   }

capy::run version:

   task<> f() {
     auto image = co_await load_image();
     capy::run(main_thread_executor)([image = std::move(image)] {
       upload_texture_to_opengl(image);
     });
   }

resume_on version:

   task<> f() {
     auto image = co_await load_image();
     resume_on(main_thread_executor);
     upload_texture_to_opengl(image);
   }

Can you see the similarity of the first two, and how the third one differs?

>> The second escape hatch is immediate_executor. It looks something
>> like this:
>>
>>     class immediate_executor {
>>     public:
>>       std::coroutine_handle<> ce.dispatch(capy::continuation &c) {
>>         // Obey the letter of the law by not just returning c.h...
>>         this->post(c);
>>         return {};
>>       }
>>
>>       void post(capy::continuation &c) {
>>         // ...but violate the spirit of the law by calling h.resume().
>>         c.h.resume();
>>       }
>>       // ...other functions here...
>>     };
> 
> How do you propose not overflowing the stack?

Well, first off my dispatch function is wrong in practice.  It should 
always return c.h instead of posting, theoretical correctness be damned. 
  At the very least it should support symmetric transfer where other 
executors support symmetric transfer.  That reduces the stack use 
dramatically.

Secondly, every function in general, and every (potentially) recursive 
function in particular, can overflow the stack, but most don't in 
practice.  In practice, my immediate_executor does not lead to unbounded 
stack growth because the IoAwaitables it uses will reset the stack.  It 
*can*, in *theory*, lead to unbounded stack growth, if it contains a 
loop that co_awaits at least one IoAwaitable that grows the stack and no 
IoAwaitables that reset the stack.  With `dispatch` fixed to support 
symmetric transfer, I'm having trouble thinking of even a contrived 
example where this actually happens.  I suppose if I had two of these 
executors, that compared unequal to each other and therefore prevented 
symmetric transfer, and I bounced a coroutine by recursively co_awaiting 
capy::run from one executor to the other, it could happen.

(Background information: I actually have years of experience with 
coroutines in a scripting language implemented in C++, but little 
experience with actual C++ coroutines.  This has definitely colored my 
expectations about coroutines.  My coroutine system had no concept of 
executors.  Everything ran on the main thread, so hopping between 
threads was never an issue.  Everything was run as if on an 
immediate_executor (but I did have and use symmetric transfer).  And it 
worked, and I never ran into a stack overflow problem related to 
coroutines.  I am speaking out of ignorance where Capy is concerned, but 
I am speaking from experience where coroutines in a more general sense 
are concerned.)

> Your `post` calls `c.h.resume()` directly. That resumes a coroutine from
> inside another coroutine's execution context. Each `resume()` adds a stack
> frame.

Until it hits an IoAwaitable that suspends the coroutine or the end of 
the coroutine, both of which unwind the stack again.

But you're absolutely right, I could have and should have used symmetric 
transfer.

> Beyond stack overflow, there are further constraints. `async_mutex` stop
> callbacks call `executor.post(cont_)` from arbitrary threads. An
> immediate_executor
> that inlines `post` would resume the coroutine on the wrong thread,
> corrupting the thread-local frame allocator. This is, ironically, exactly
> the class of bug you asked us to document in condition 4.

OK, so it actually doesn't work.  It's not just a potentially bad idea, 
it's undefined behavior.  That's exactly the information I was looking 
for (ideally in the documentation, not just the mailing list).

Can this be fixed?  Can immediate_executor temporarily load the correct 
allocator into TLS, call coroutine_handle::resume, and restore the 
previous allocator when coroutine_handle::resume returns?

Running on the "wrong" thread is the whole point of immediate_executor, 
but it should not invoke undefined behavior.


-- 
Rainer Deyke - [email protected]

_______________________________________________
Boost mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.boost.org/mailman3/lists/boost.lists.boost.org/
Archived at: https://lists.boost.org/archives/list/[email protected]/message/WALHSW32XADFVOQBZ4SUJRUTT5LDGL7S/