Slow performance using multi_curl_* with HTTPs requests

--===============2017259814==
Content-Type: multipart/alternative; boundary="00000000000067071805c56ad9c3"

--00000000000067071805c56ad9c3
Content-Type: text/plain; charset="UTF-8"

Hi,

I am experiencing performance issues using multi curl combined with HTTPs
requests on a lately updated OS. It seems only related to multi curl and
HTTPs. Single, serialized, curl requests with HTTPs do not show any slow
down. So it seems not to be related to e.g. the OpenSSL version or the
SSL/TLS Handshake. I am not quite sure if this is related to the PHP side
of the multi curl implementation or libcurl multi curl interface...

I ran some tests to track the problem down but until now without any
success.

The tests I did do either 100 requests in a row (no multi curl; script name
curl_plain.php) or 100 times 10 requests in parallel with multi curl
(curl_multi.php). Both call a PHP script just sleeping for 1 second. One
time I called it via HTTP the other time via HTTPs. I ran all the tests on
both nodes. One node is on Debian Jessie (the old node). The other one is
on Debian Stretch (the new node). The following run times are all above 100
seconds since it can't get faster with a sleep of 1 second called 100
times. The interesting part is the overhead above the 100 seconds.

The slow down happened with the upgrade from Jessie to Stretch. I tested it
with Buster and with PHP7.4, as well. All combinations show the same issue.
So it does not go away with any further upgrade.

Since I am heavily relying on a performant multi_curl for some processes it
is necessary for me to resolve the performance issue.

The old node:
Debian Jessie (8.6)
PHP 7.1.17-1+0~20180505045956.17+jessie~1.gbpde69c6
OpenSSL 1.0.1t
libcurl(3) 7.38.0-4+deb8u4

The new node:
Debian Stretch (9.13)
PHP 7.1.33-37+0~20210501.55+debian9~1.gbpcb9127
OpenSSL 1.1.0l
libcurl(3) 7.52.1-5+deb9u13

Both nodes are exactly the same hardware wise. It is a dual CPU system with
NUMA activated to share the memory between both sockets. Both nodes are
located in the same network and call, of course, the same script on another
node in a different network.

The tests with HTTP:
old $ php curl_plain.php
   Total: 101.47104287148 seconds
old $ php curl_multi.php
   Total: 101.5175549984 seconds
new $ php curl_plain.php
   Total: 101.44491195679 seconds
new $ php curl_multi.php
   Total: 101.5433011055 seconds
-> no issue

The tests with HTTPs:
old$ php src/curl_plain.php
  Total: 103.15611720085 seconds
old$ php src/curl_multi.php
  Total: 104.17670083046 seconds
new$ php src/curl_plain.php
  Total: 103.91886115074 seconds
new$ php src/curl_multi.php
  Total: 110.56924200058 seconds
-> It makes sense that with multi_curl there is a little bit more overhead
but the big gap on the new node is a little bit too much I would say.

Does anyone have an idea what the issue could be?

Kind regards
Thomas

--00000000000067071805c56ad9c3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<div><br></div><div>I am experiencing performance issue=
s using multi curl combined with HTTPs requests on a lately updated OS. It =
seems only related to multi curl and HTTPs. Single, serialized, curl reques=
ts with HTTPs do not show any slow down. So it seems not to be related to e=
.g. the OpenSSL version or the SSL/TLS Handshake. I am not quite sure if th=
is is related to the PHP side of the multi curl implementation or libcurl m=
ulti curl interface...</div><div><br></div><div>I ran some tests to track t=
he problem down but until now without any success.=C2=A0</div><div><br></di=
v><div>The tests I did do either 100 requests in a row (no multi curl; scri=
pt name curl_plain.php) or 100 times 10 requests in parallel with multi cur=
l (curl_multi.php). Both call a PHP script just sleeping for 1 second. One =
time I called it via HTTP the other time via HTTPs. I ran all the tests on =
both nodes. One node is on Debian Jessie (the old node). The other one is o=
n Debian Stretch (the new node). The following run times are all above 100 =
seconds since it can&#39;t get faster with a sleep of 1 second called 100 t=
imes. The interesting part is the overhead above the 100 seconds.</div><div=
><br></div><div>The slow down happened with the upgrade from Jessie to Stre=
tch. I tested it with Buster and with PHP7.4, as well. All combinations sho=
w the same issue. So it does not go away with any further upgrade.</div><di=
v><br></div><div>Since I am heavily relying on a performant multi_curl for =
some processes it is necessary for me to resolve the performance issue.</di=
v><div><br></div><div>The old node:<br></div><div>Debian Jessie (8.6)</div>=
<div>PHP 7.1.17-1+0~20180505045956.17+jessie~1.gbpde69c6<br></div><div>Open=
SSL 1.0.1t<br></div><div>libcurl(3) 7.38.0-4+deb8u4<br></div><div><br></div=
><div>The new node:</div><div>Debian Stretch (9.13)</div><div>PHP 7.1.33-37=
+0~20210501.55+debian9~1.gbpcb9127<br></div><div>OpenSSL 1.1.0l<br></div><d=
iv>libcurl(3)=C2=A07.52.1-5+deb9u13</div><div><br></div><div>Both nodes are=
 exactly the same hardware wise. It is a dual CPU system with NUMA activate=
d to share the memory between both sockets. Both nodes are located in the s=
ame network and call, of course, the same script on another node in a diffe=
rent network.</div><div><br></div><div>The tests with HTTP:</div><div>old $=
 php curl_plain.php<br>=C2=A0 =C2=A0Total: 101.47104287148 seconds<br>old $=
 php curl_multi.php<br>=C2=A0 =C2=A0Total: 101.5175549984 seconds<br>new $ =
php curl_plain.php<br>=C2=A0 =C2=A0Total: 101.44491195679 seconds<br>new $ =
php curl_multi.php<br>=C2=A0 =C2=A0Total: 101.5433011055 seconds<br></div><=
div>-&gt; no issue</div><div><br></div><div>The tests with HTTPs:</div><div=
>old$ php src/curl_plain.php<br>=C2=A0 Total: 103.15611720085 seconds<br>ol=
d$ php src/curl_multi.php<br>=C2=A0 Total: 104.17670083046 seconds<br></div=
><div>new$ php src/curl_plain.php<br>=C2=A0 Total: 103.91886115074 seconds<=
br>new$ php src/curl_multi.php<br>=C2=A0 Total: 110.56924200058 seconds<br>=
</div><div>-&gt; It makes sense that with multi_curl there is a little bit =
more overhead but the big gap on the new node is a little bit too much I wo=
uld say.</div><div><br></div><div>Does anyone have an idea what the issue c=
ould be?</div><div><br></div><div>Kind regards</div><div>Thomas</div></div>

--00000000000067071805c56ad9c3--

--===============2017259814==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: base64
Content-Disposition: inline

X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaHR0cHM6Ly9j
b29sLmhheHguc2UvY2dpLWJpbi9tYWlsbWFuL2xpc3RpbmZvL2N1cmwtYW5kLXBocAo=

--===============2017259814==--