Re: Help with ZEO performance

--000000000000f462b7057b83830a
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, Nov 22, 2018 at 10:39 AM Colin Garriga-Sala=C3=BCn <[email protected]> =
wrote:

> ZEO 5.2.0 with Python2.7
>

So some cheap things to try (not necessarily in order):

   - Run the server on PyPy.
   See the graphs at the end of:
   https://github.com/jimfulton/byteserver/wiki/performance-2016-10-26

   I'm not sure this will help you, given that writing seems to be slower,
   but it might be worth a try.

   - Upgrade to Python 3 (if practical).
      - I was hoping you could use a Python 3 storage server with Python 2
      clients if you configured it to use msgpack, but alas, this is not th=
e
      case. I'm working on trying to make this possible.

      - Use biggish persistent ZEO caches, if you aren't already.
   - Maybe use bigger object caches.
   - Maybe a faster storage server machine.
   - Double check that you aren't saturating network communication to the
   storage server. That was something we had to deal with at ZC.

Some more involved things to try:

   - If you can arrange for read-only servers (maybe with clever
   load-balancer rules), then use ZRS secondaries for these. You've already
   dismissed this, but maybe think harder about making it work.
   - Maybe use fewer clients. For example, if you're CPU bound and using a
   cloud provider, maybe use faster  machines.
      - If you're not CPU bound, then maybe use fewer clients and more
      threads per client.
   - Maybe try a ZEO fan-out configuration, where you have multiple ZEO
   servers that wrap ZEO clients of a single ZEO server.  I'm not aware of
   anyone who's run this in production. :)

There are probably other options depending on your application. :)

Jim




>
> On 22/11/2018 17:46, Jim Fulton wrote:
> > That's a lot of connections.  At ZC (Zope Corporation) I think we got
> > up to ~80 connections.
> >
> > What version of ZEO are you using?
>
> --
> You received this message because you are subscribed to the Google Groups
> "zodb" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>


--=20
Jim Fulton
http://jimfulton.info

--=20
You received this message because you are subscribed to the Google Groups "=
zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to [email protected].
For more options, visit https://groups.google.com/d/optout.

--000000000000f462b7057b83830a
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div d=
ir=3D"ltr">On Thu, Nov 22, 2018 at 10:39 AM Colin Garriga-Sala=C3=BCn &lt;<=
a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&gt; wro=
te:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">ZEO 5.2.0 wi=
th Python2.7<br></blockquote><div><br></div><div>So some cheap things to tr=
y (not necessarily in order):</div><div><ul><li>Run the server on PyPy.<br>=
See the graphs at the end of:=C2=A0<a href=3D"https://github.com/jimfulton/=
byteserver/wiki/performance-2016-10-26" target=3D"_blank">https://github.co=
m/jimfulton/byteserver/wiki/performance-2016-10-26</a><br><br>I&#39;m not s=
ure this will help you, given that writing seems to be slower, but it might=
 be worth a try.<br><br></li><li>Upgrade to Python 3 (if practical).</li><u=
l><li>I was hoping you could use a Python 3 storage server with Python 2 cl=
ients if you configured it to use msgpack, but alas, this is not the case. =
I&#39;m working on trying to make this possible.<br><br></li></ul><li>Use b=
iggish persistent ZEO caches, if you aren&#39;t already.</li><li>Maybe use =
bigger object caches.</li><li>Maybe a faster storage server machine.</li><l=
i>Double check that you aren&#39;t saturating network communication to the =
storage server. That was something we had to deal with at ZC.</li></ul>Some=
 more involved things to try:</div><div><ul><li>If you can arrange for read=
-only servers (maybe with clever load-balancer rules), then use ZRS seconda=
ries for these. You&#39;ve already dismissed this, but maybe think harder a=
bout making it work.</li><li>Maybe use fewer clients. For example, if you&#=
39;re CPU bound and using a cloud provider, maybe use faster=C2=A0 machines=
.</li><ul><li>If you&#39;re not CPU bound, then maybe use fewer clients and=
 more threads per client.</li></ul><li>Maybe try a ZEO fan-out configuratio=
n, where you have multiple ZEO servers that wrap ZEO clients of a single ZE=
O server.=C2=A0 I&#39;m not aware of anyone who&#39;s run this in productio=
n. :)</li></ul>There are probably other options depending on your applicati=
on. :)</div><div><br></div><div>Jim</div><div><br></div><div><br></div><div=
>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
On 22/11/2018 17:46, Jim Fulton wrote:<br>
&gt; That&#39;s a lot of connections.=C2=A0 At ZC (Zope Corporation) I thin=
k we got<br>
&gt; up to ~80 connections.<br>
&gt;<br>
&gt; What version of ZEO are=C2=A0you using?<br>
<br>
-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:zodb%[email protected]" target=3D"_b=
lank">[email protected]</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" rel=
=3D"noreferrer" target=3D"_blank">https://groups.google.com/d/optout</a>.<b=
r>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
 class=3D"m_3534548400388646136gmail-m_-3614220510014838862gmail_signature"=
>Jim Fulton<br><a href=3D"http://jimfulton.info" target=3D"_blank">http://j=
imfulton.info</a><br></div></div></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">zodb+unsubscri=
[email protected]</a>.<br />
For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
s://groups.google.com/d/optout</a>.<br />

--000000000000f462b7057b83830a--