[SPDK] Re: Performance impact with "enable_zerocopy_send"

Hi Alexey,

Looks like you are right that ZEROCOPY is not effective for smaller buffer
sizes like 4KB.

Here are the performance changes I got after enabling ZEROCOPY

Random Read in 4KB:                      Dropped  from 5042 KIOPS to 3080
KIOPS; -38.91%
Random Write in 4KB:                      Dropped  from 6301 KIOPS to 5899
KIOPS; -6.38%

Sequential Read in 128KB:               Increased  from 158.3 KIOPS to 262
KIOPS; +65.51%
Sequential Write in 128KB:               Increased  from 161.6 KIOPS to
167.5 KIOPS; +3.65%

Why does XEROCOPY have a much bigger impact on Read than on Write?

Thanks,
Charlie

On Mon, Feb 8, 2021 at 11:37 AM Alexey Marchuk <alexeymar(a)nvidia.com> wrote:

> Hello Charlie,
>
> Could you add more details about the performance drop? Are you using SPDK
> target and/or initiator and what is payload size?
> I would expect some degradation with small payloads, as written it this
> doc
> https://www.kernel.org/doc/html/v4.15/networking/msg_zerocopy.html#opportunity-and-caveats
> :
> "MSG_ZEROCOPY is generally only effective at writes over around 10 KB. "
>
> Best regards,
> Alexey Marchuk
>
> -----Original Message-----
> From: Charlie Li <charlie.li(a)gmail.com>
> Sent: Wednesday, January 20, 2021 12:41 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Performance impact with "enable_zerocopy_send"
>
> Hello,
>
> In SPDK 20.10, "enable_zerocopy_send" is set to "false" by default.
>
> I tried to set  "enable_zerocopy_send" to true, but noticed the
> performance dropped.
>
> Is this as expected? My understanding is that setting
> "enable_zerocopy_send" to true should improve performance.
>
> Thanks,
> Charlie
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>