[Capy/Corosio review] Benchmark results

Hi, some months ago I started implementing new benchmarks for
Boost.Redis to compare it to the current state of other popular
clients.  While underway with it Boost.Redis gained Corosio support,
something that has been announced by Ruben Perez in this mailing list
recently.

In this email I would like tol share the results I obtained for
appreciation by the Capy/Corosio review audience. The code is public
and available at https://github.com/mzimbres/redis-cli-comp.

The new benchmarks simulate a scenario where Redis is mostly used,
namely, internet facing servers (usually HTTP) that serve connections
concurrently while receiving server pushes e.g. pubsub events. It
consists of

  1. starting multiple independent sessions that issue commands in a loop.
  2. subscribing to a channel to receive pubsub events.

## Runtime Performance

The metric used to assess the runtime performance was the wall-clock
multiplied by %CPU consumption used by the client. This metric takes
into account that clients might use a different number of threads. The
following result were obtained (lower is better)

    Client                             Time x %CPU (nomalized)
    -------------------------------------------------------------------
    boost_redis_corosio                    1.00
    boost_redis_asio_cb                   1.35
    boost_redis_asio_co                  1.73
    redis_rs (Rust)                            4.62
    go_redis (Go)                            22.33

The thread and file-descriptor usage by each client was

    Client                         threads      fd-nr
    --------------------------------------------------
    boost-redis-corosio         3             7
    boost-redis-asio-co         2             7
    boost-redis-asio-cb         2             7
    redis-rs                            1           10
    go-redis                         24       1006

## Application build time

This is the time taken to build the benchmark program excluding the
time taken to build the client library

    Library                       Time (s)
    ---------------------------------------
    boost-redis-corosio      1.68
    boost-redis-asio-co      3.60
    boost-redis-asio-cb      8.62
    redis-rs                         3.10
    go-redis                        0.28

## Client build time

Here we see the time taken to build only the client

     Library                           Time (s)
     -------------------------------------------
     boost-redis-lib-corosio       5.3
     boost-redis-lib-asio          21.7
     redis-rs                              9.6

Note: I haven't found a way to measure this for go clients.

## Total build time

The times below include the build time of all dependencies (which have
been previously downloaded)

    Client                         Time (s)
    ---------------------------------------
    boost-redis-corosio      33.6
    boost-redis-asio-co      35.5
    boost-redis-asio-cb      40.5
    redis-rs                         42.6
    go-redis                          9.0

## Executable size

That is the size of release builds of each benchmark program.

    Executable                   Size
    --------------------------------------
    boost_redis_corosio     1.5M
    boost_redis_asio_cb    1.9M
    boost_redis_asio_co    1.3M
    redis_rs                        2.0M
    go_redis                       8.2M

## Summary

It is not the point of this report to analyse the results in detail or
emit any verdict, Corosio however has come out with really impressive
results IMO.

Marcelo
_______________________________________________
Boost mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://lists.boost.org/mailman3/lists/boost.lists.boost.org/
Archived at: https://lists.boost.org/archives/list/[email protected]/message/DRXACCXHE4RCRQHPPVXI56JT2V7CJMFD/