[SPDK] Re: Performance of spdk_trace

--===============6535631120628816288==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

at 5:44 AM, Harris, James R <james.r.harris(a)intel.com> wrote:

> Hi Jinhao,
> =

> I=E2=80=99m not aware of any detailed benchmarking, but it can certainly =
support the trace event rates that you describe.
> =

> You can use the null bdev w/ the bdevperf application as a rough approxim=
ation.  Each bdev IO generates two trace events (one for IO start, one for =
IO end), so we can use the null bdev as an easy test to generate as many of=
 those trace events as possible.
> =

> First put the following in a file named null.json:
> =

> {
>  "subsystems": [
>    {
>      "subsystem": "bdev",
>      "config": [
>        {
>          "method": "bdev_null_create",
>          "params": {
>            "name": "null0",
>            "num_blocks": 2097152,
>            "block_size": 512
>          }
>        }
>      ]
>    }
>  ]
> }
> =

> Then run the bdevperf app without events:
> =

> test/bdev/bdevperf/bdevperf -q 4 -o 1024 -w randread -t 5 -c null.json
> =

> In one of my development VMs, I see 20M IO/s.  This is roughly 50ns per I=
O.
> =

> Next run it with bdev trace events enabled by adding the =E2=80=9C-e bdev=
=E2=80=9D command line option:
> =

> test/bdev/bdevperf/bdevperf -q 4 -o 1024 -w randread -t 5 -c null.json -e=
 bdev
> =

> In the same VM, I see 14M IO/s.  This is roughly 70ns per IO.  That=E2=80=
=99s 20ns extra overhead for 2 trace events.
> =

> It scales to multiple cores as well =E2=80=93 add =E2=80=9C-m 0xF -C=E2=
=80=9D to both bdevperf command lines and it will do the same test on 4 cor=
es at once with roughly that same 20M v. 14M IO/s on each of the 4 cores.

That=E2=80=99s awesome performance. Thanks!

> =

> Of course your platform will have different hardware, so results will var=
y somewhat.  The rate limit will also depend on the number and size of the =
trace arguments for each trace event.
> =

> Best regards,
> =

> Jim Harris
> =

> =

> =

> =

> From: Jinhao Fan <fanjinhao21s(a)ict.ac.cn>
> Date: Sunday, August 28, 2022 at 6:14 PM
> To: spdk(a)lists.01.org <spdk(a)lists.01.org>
> Subject: [SPDK] Performance of spdk_trace
> Hi there,
> =

> During my daily use I found the tracing system of SPDK and spdk_trace_rec=
ord
> very low overhead. So I=E2=80=99m planning to port to other applications =
where it
> might generate several 100K events per second. I think such a speed should
> be normal for SPDK applications. Have you ever measured how many events p=
er
> second is spdk_trace able to handle?
> =

> Thanks!
> =

> Jinhao Fan
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org



--===============6535631120628816288==--