Re: [Capy/Corosio review] Benchmark results

On Sun, Jun 28, 2026 at 3:01 PM Marcelo Zimbres Silva via Boost <
[email protected]> wrote:

> Hi, some months ago I started implementing new benchmarks for
> Boost.Redis to compare it to the current state of other popular
> clients.  While underway with it Boost.Redis gained Corosio support,
> something that has been announced by Ruben Perez in this mailing list
> recently.
>
> In this email I would like tol share the results I obtained for
> appreciation by the Capy/Corosio review audience. The code is public
> and available at https://github.com/mzimbres/redis-cli-comp.
>
> The new benchmarks simulate a scenario where Redis is mostly used,
> namely, internet facing servers (usually HTTP) that serve connections
> concurrently while receiving server pushes e.g. pubsub events. It
> consists of
>
>   1. starting multiple independent sessions that issue commands in a loop.
>   2. subscribing to a channel to receive pubsub events.
>
> ## Runtime Performance
>
> The metric used to assess the runtime performance was the wall-clock
> multiplied by %CPU consumption used by the client. This metric takes
> into account that clients might use a different number of threads. The
> following result were obtained (lower is better)
>
>     Client                             Time x %CPU (nomalized)
>     -------------------------------------------------------------------
>     boost_redis_corosio                    1.00
>     boost_redis_asio_cb                   1.35
>     boost_redis_asio_co                  1.73
>     redis_rs (Rust)                            4.62
>     go_redis (Go)                            22.33
>

These results seem a little suspicious to me. Rust is 4.62x slower? C, C++
and Rust are relatively the same perf-wise, because they all just compile
to LLVM IR and get turned into a final executable by clang.

I've pointed this out before but the version of Tokio being tested is ~20
versions behind.

I think you should publish the flamegraphs here, because there's something
wrong if a language that compiles to llvm ir is over 4.5x slower, and this
is worthy of a little skepticism.

Being 20x faster than Go also sounds kind of suspicious, if I'm being
honest.

redis-rs sits at around 80 million total downloads (
https://crates.io/crates/redis) and has numerous contributors.

Something is going on here, and only the flamegraphs can really tell us.

It could very well be that the crate isn't well-implemented, of course.

For the record, you can see the latest stable version of a crate here:
https://crates.io/crates/tokio

- Christian
_______________________________________________
Boost mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.boost.org/mailman3/lists/boost.lists.boost.org/
Archived at: https://lists.boost.org/archives/list/[email protected]/message/ERR44FUI4ZGDEKEV6LRXIDJELPTH7V4R/