Re: Help with ZEO performance

--0000000000009b7bfc057b8c50a1
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Jim,
After a small email discussion with you (this was a long long time ago...),
I'm using a fan-out configuration in production since more than ten years!
Because of a slow network connection between the main ZEO server which is
storing our CMS contents, and our management servers which are hosted in
our local network, this kind of configuration was really helpful to keep
good performances!!
I've written a small topic on this configuration (
http://www.ztfy.org/articles-et-tutoriels/2010/09/installation-dun-serveur-=
zeo-en-mode.html
).
Best regards,
Thierry


Le dim. 25 nov. 2018 =C3=A0 22:01, Jim Fulton <[email protected]> a =C3=A9=
crit :

>
>
> On Thu, Nov 22, 2018 at 10:39 AM Colin Garriga-Sala=C3=BCn <[email protected]=
>
> wrote:
>
>> ZEO 5.2.0 with Python2.7
>>
>
> So some cheap things to try (not necessarily in order):
>
>    - Run the server on PyPy.
>    See the graphs at the end of:
>    https://github.com/jimfulton/byteserver/wiki/performance-2016-10-26
>
>    I'm not sure this will help you, given that writing seems to be
>    slower, but it might be worth a try.
>
>    - Upgrade to Python 3 (if practical).
>       - I was hoping you could use a Python 3 storage server with Python
>       2 clients if you configured it to use msgpack, but alas, this is no=
t the
>       case. I'm working on trying to make this possible.
>
>       - Use biggish persistent ZEO caches, if you aren't already.
>    - Maybe use bigger object caches.
>    - Maybe a faster storage server machine.
>    - Double check that you aren't saturating network communication to the
>    storage server. That was something we had to deal with at ZC.
>
> Some more involved things to try:
>
>    - If you can arrange for read-only servers (maybe with clever
>    load-balancer rules), then use ZRS secondaries for these. You've alrea=
dy
>    dismissed this, but maybe think harder about making it work.
>    - Maybe use fewer clients. For example, if you're CPU bound and using
>    a cloud provider, maybe use faster  machines.
>       - If you're not CPU bound, then maybe use fewer clients and more
>       threads per client.
>    - Maybe try a ZEO fan-out configuration, where you have multiple ZEO
>    servers that wrap ZEO clients of a single ZEO server.  I'm not aware o=
f
>    anyone who's run this in production. :)
>
> There are probably other options depending on your application. :)
>
> Jim
>
>
>
>
>>
>> On 22/11/2018 17:46, Jim Fulton wrote:
>> > That's a lot of connections.  At ZC (Zope Corporation) I think we got
>> > up to ~80 connections.
>> >
>> > What version of ZEO are you using?
>>
>> --
>> You received this message because you are subscribed to the Google Group=
s
>> "zodb" group.
>> To unsubscribe from this group and stop receiving emails from it, send a=
n
>> email to [email protected].
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> Jim Fulton
> http://jimfulton.info
>
> --
> You received this message because you are subscribed to the Google Groups
> "zodb" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> For more options, visit https://groups.google.com/d/optout.
>


--=20
http://www.imagesdusport.com -- http://pyams.readthedocs.io

--=20
You received this message because you are subscribed to the Google Groups "=
zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to [email protected].
For more options, visit https://groups.google.com/d/optout.

--0000000000009b7bfc057b8c50a1
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon=
t-family:monospace,monospace;color:rgb(120,63,4)">Hi Jim,</div><div class=
=3D"gmail_default" style=3D"font-family:monospace,monospace;color:rgb(120,6=
3,4)">After a small email discussion with you (this was a long long time ag=
o...), I&#39;m using a fan-out configuration in production since more than =
ten years!</div><div class=3D"gmail_default" style=3D"font-family:monospace=
,monospace;color:rgb(120,63,4)">Because of a slow network connection betwee=
n the main ZEO server which is storing our CMS contents, and our management=
 servers which are hosted in our local network, this kind of configuration =
was really helpful to keep good performances!!</div><div class=3D"gmail_def=
ault" style=3D"font-family:monospace,monospace;color:rgb(120,63,4)">I&#39;v=
e written a small topic on this configuration (<a href=3D"http://www.ztfy.o=
rg/articles-et-tutoriels/2010/09/installation-dun-serveur-zeo-en-mode.html"=
>http://www.ztfy.org/articles-et-tutoriels/2010/09/installation-dun-serveur=
-zeo-en-mode.html</a>).</div><div class=3D"gmail_default" style=3D"font-fam=
ily:monospace,monospace;color:rgb(120,63,4)">Best regards,</div><div class=
=3D"gmail_default" style=3D"font-family:monospace,monospace;color:rgb(120,6=
3,4)">Thierry</div><div class=3D"gmail_default" style=3D"font-family:monosp=
ace,monospace;color:rgb(120,63,4)"><br></div></div></div><br><div class=3D"=
gmail_quote"><div dir=3D"ltr">Le=C2=A0dim. 25 nov. 2018 =C3=A0=C2=A022:01, =
Jim Fulton &lt;<a href=3D"mailto:[email protected]">[email protected]</a>=
&gt; a =C3=A9crit=C2=A0:<br></div><blockquote class=3D"gmail_quote" style=
=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=
=3D"ltr"><div dir=3D"ltr"><br><br><div class=3D"gmail_quote"><div dir=3D"lt=
r">On Thu, Nov 22, 2018 at 10:39 AM Colin Garriga-Sala=C3=BCn &lt;<a href=
=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&gt; wrote:<br=
></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;=
border-left:1px solid rgb(204,204,204);padding-left:1ex">ZEO 5.2.0 with Pyt=
hon2.7<br></blockquote><div><br></div><div>So some cheap things to try (not=
 necessarily in order):</div><div><ul><li>Run the server on PyPy.<br>See th=
e graphs at the end of:=C2=A0<a href=3D"https://github.com/jimfulton/bytese=
rver/wiki/performance-2016-10-26" target=3D"_blank">https://github.com/jimf=
ulton/byteserver/wiki/performance-2016-10-26</a><br><br>I&#39;m not sure th=
is will help you, given that writing seems to be slower, but it might be wo=
rth a try.<br><br></li><li>Upgrade to Python 3 (if practical).</li><ul><li>=
I was hoping you could use a Python 3 storage server with Python 2 clients =
if you configured it to use msgpack, but alas, this is not the case. I&#39;=
m working on trying to make this possible.<br><br></li></ul><li>Use biggish=
 persistent ZEO caches, if you aren&#39;t already.</li><li>Maybe use bigger=
 object caches.</li><li>Maybe a faster storage server machine.</li><li>Doub=
le check that you aren&#39;t saturating network communication to the storag=
e server. That was something we had to deal with at ZC.</li></ul>Some more =
involved things to try:</div><div><ul><li>If you can arrange for read-only =
servers (maybe with clever load-balancer rules), then use ZRS secondaries f=
or these. You&#39;ve already dismissed this, but maybe think harder about m=
aking it work.</li><li>Maybe use fewer clients. For example, if you&#39;re =
CPU bound and using a cloud provider, maybe use faster=C2=A0 machines.</li>=
<ul><li>If you&#39;re not CPU bound, then maybe use fewer clients and more =
threads per client.</li></ul><li>Maybe try a ZEO fan-out configuration, whe=
re you have multiple ZEO servers that wrap ZEO clients of a single ZEO serv=
er.=C2=A0 I&#39;m not aware of anyone who&#39;s run this in production. :)<=
/li></ul>There are probably other options depending on your application. :)=
</div><div><br></div><div>Jim</div><div><br></div><div><br></div><div>=C2=
=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e=
x;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
On 22/11/2018 17:46, Jim Fulton wrote:<br>
&gt; That&#39;s a lot of connections.=C2=A0 At ZC (Zope Corporation) I thin=
k we got<br>
&gt; up to ~80 connections.<br>
&gt;<br>
&gt; What version of ZEO are=C2=A0you using?<br>
<br>
-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:zodb%[email protected]" target=3D"_b=
lank">[email protected]</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" rel=
=3D"noreferrer" target=3D"_blank">https://groups.google.com/d/optout</a>.<b=
r>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
 class=3D"m_-2060712074542887746m_3534548400388646136gmail-m_-3614220510014=
838862gmail_signature">Jim Fulton<br><a href=3D"http://jimfulton.info" targ=
et=3D"_blank">http://jimfulton.info</a><br></div></div></div>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]" target=3D"_bla=
nk">[email protected]</a>.<br>
For more options, visit <a href=3D"https://groups.google.com/d/optout" targ=
et=3D"_blank">https://groups.google.com/d/optout</a>.<br>
</blockquote></div><br clear=3D"all"><div><br></div>-- <br><div dir=3D"ltr"=
 class=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"l=
tr"><div><a href=3D"http://www.imagesdusport.com" target=3D"_blank">http://=
www.imagesdusport.com</a> -- <a href=3D"http://pyams.readthedocs.io" target=
=3D"_blank">http://pyams.readthedocs.io</a></div></div></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;zodb&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">zodb+unsubscri=
[email protected]</a>.<br />
For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
s://groups.google.com/d/optout</a>.<br />

--0000000000009b7bfc057b8c50a1--