Re: Difference in active/non active objects distribution in caches of zserver-threads between Plone 3/Plone 4

I really enjoy always these posts from Hanno; thank you very much for 
explaining this kind of things to all of us, mere mortals :-)

I want to make a clarification: Plone 3.0 used Zope 2.10.4 and not "an 
older version" as Hanno states in his answer.

http://plone.org/products/plone/releases/3.0

perhaps what Fred mentions is more related with the Python version 
supported at that time (2.4.4)?

for years I've heard terror stories about Python 2.4 memory leaks so 
today I just checked the release changelog and found that the last bug 
fix release of that branch (2.4.6 ) included something that seems to be 
of interest:

- Issue #2620: Overflow checking when allocating or reallocating memory
   was not always being done properly in some python types and extension
   modules.  PyMem_MALLOC, PyMem_REALLOC, PyMem_NEW and PyMem_RESIZE have
   all been updated to perform better checks and places in the code that
   would previously leak memory on the error path when such an allocation
   failed have been fixed.

http://www.python.org/download/releases/2.4.6/NEWS.txt

being a bug release seems to me there is no reason not to upgrade.

as a bonus, and for the hopeless optimist inside each one of us, I found 
this post from 5 years ago:

http://blog.hannosch.eu/2008/12/plone-versus-python-3.html

happy new year!

Héctor Velarde

On 22-10-2013 10:20, Hanno Schlichting wrote:
> Fred van Dijk <f.van.dijk@...> writes:
>> It seems that one thread is 'preferred' and gets most of the traffic and
>> fills up its cache, where thread 2 and 3 are barely used, at least looking
>> at the cache values.
>
> I'm guessing the answer to this is two-fold. First there was a ZODB level
> optimization first introduced in ZODB 3.9. Plone 3 used an older version and
> only Plone 4.0 upgraded to ZODB 3.9 (and 4.1+ upgraded to ZODB 3.10).
>
> The relevant entry from the ZODB 3.9.0 changelog:
>
> "Previously, database connections were managed as a stack. This tended to
> cause the same connection(s) to be used over and over. For example, the most
> used connection would typically be the only connection used. In some rare
> situations, extra connections could be opened and end up on the top of the
> stack, causing extreme memory wastage. Now, when connections are placed on
> the stack, they sink below existing connections that have more active objects."
>
> The second part to this story is Plone 4 indeed being much faster.
>
> With Plone being so much faster, it's most often able to completely handle
> a request, before any second request gets sent to the same server. This means
> there's in practice only ever one thread actually active and this thread uses
> the same ZODB connection over and over again.
>
> In the more uncommon scenario, where indeed two requests are handled in
> parallel by two threads, each of them will use a different ZODB connection.
>
> In Plone 3 both the often used or less used connection could end up on top of
> the connection pool stack. Effectively leading to both (or multiple)
> connections to be used and their caches to grow. In Plone 4 the connection
> with the largest cache is always returning to the top of the connection pool
> and will be used most of the time.
>
> This theory is unproven, so I might be wrong :)
>
>> Increasing zserver-threads in Plone 4 to say 6 only lists more thread
>> caches that are not used, 200-500 objects active at most.
>
> There are only very few types of requests that can actually be executed in
> parallel by one Zope process. Except for I/O, Python doesn't actually execute
> threads in parallel, but there's always just one thread active at a time.
>
> All the benchmarks I've seen suggest that using either one or two threads
> is best for performance and the zope2instance recipe defaults to 2 threads.
> If you need to handle more concurrent load, you need to add more processes.
>
> Hope this helps,
> Hanno
>
>
> ------------------------------------------------------------------------------
> October Webinars: Code for Performance
> Free Intel webinars can help you accelerate application performance.
> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
> the latest Intel processors and coprocessors. See abstracts and register >
> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
> _______________________________________________
> Plone-developers mailing list
> Plone-developers-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
> https://lists.sourceforge.net/lists/listinfo/plone-developers
>

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT 
organizations don't have a clear picture of how application performance 
affects their revenue. With AppDynamics, you get 100% visibility into your 
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk

_______________________________________________
Plone-developers mailing list
Plone-developers-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/plone-developers