Re: Re: overhead of ZEO connection

Thank you very much and fully understand your explanation given here. To 
take care of GIL issue I have divided my application into multiple 
processes or more precisely you can say that multiple zope instances.
Now each zope instance can have multiple connections.
In this case I want to make sure that my ZEO wont become the bottleneck in 
the system.
What recommendations can you guys suggest to take care of this issue ?
On higher side my application can have upto 40 zope instances.

Best Regards,
Sanjay

On Wednesday, 11 May 2016 18:51:35 UTC+5:30, Jim Fulton wrote:
>
> On Wed, May 11, 2016 at 8:25 AM, Sanjay Rao <[email protected] 
> <javascript:>> wrote: 
> > How many connections can be assumed to be safe in that case. ? 
> > What if my application opens lots of connections(hundreds) ? 
>
> This isn't a "safety" issue. Each (ZODB not ZEO) connection has a 
> cache, which consumes memory.  How much memory your application 
> consumes depends on how many connections you have and how many objects 
> are in the caches. Both of 
> these are tunable.  What's right is a design decision. There isn't a 
> one-size-fits-all answer. 
>
> Note that if your application is compute bound, then it doesn't make 
> sense to have many threads. For example, traditional (Web1) Zope 
> applications tended to spend most of their time computing pages.  For 
> these sorts of applications, I would a) try to make the cache large 
> enough for most or all of the needed data to fit in RAM and b) use a 
> single thread, because the GIL wouldn't allow any more throughput with 
> more threads. 
>
> Also, WRT connection pool size, in addition to warning when the number 
> of connections exceeds the pool size, excess connections will be 
> discarded when they are closed, discarding their caches. If exceeding 
> the pool size is rare, then this works well. If you routinely have 
> more connections, you should consider increasing the pool size, or 
> more likely reducing the thread count. 
>
> Jim 
>
> > 
> > On Tuesday, 10 May 2016 03:40:28 UTC+5:30, Tres Seaver wrote: 
> >> 
> >> -----BEGIN PGP SIGNED MESSAGE----- 
> >> Hash: SHA1 
> >> 
> >> On 05/08/2016 08:11 AM, Sanjay Rao wrote: 
> >> 
> >> > Thanks for reply. If pool size is 8 then is following true ? 
> >> > 
> >> > In application; there are some long running(may run more than a hour) 
> >> >  threads. there are 8 such threads these 8 long running threads get 
> >> > all available 8 connections in pool 
> >> > 
> >> > There are some short duration threads also, which read/write in 
> >> > database. These short duration threads won't get connection until at 
> >> > least one of the long running thread does not release connection. 
> >> 
> >> The DB object is willing to create new connections if the pools is 
> empty: 
> >> it will begin to warn about the overage in the log. 
> >> 
> >> 
> >> Tres. 
> >> - -- 
> >> =================================================================== 
> >> Tres Seaver          +1 540-429-0999          [email protected] 
> >> Palladion Software   "Excellence by Design"    http://palladion.com 
> >> -----BEGIN PGP SIGNATURE----- 
> >> Version: GnuPG v1 
> >> 
> >> iQIcBAEBAgAGBQJXMQq4AAoJEPKpaDSJE9HY81IQAJuyUG8iSeEfUsND+odOjdwO 
> >> 9c9wqJfJa5kAqwgrK9GpIO7e1LEGhjJuPpZ8EbDz0F75IOLNi10a0KInUtxouuqA 
> >> KyyvxsLagpytYXEeH9iBakUeQuqAp1E6eVl/ceUKiCtR97eKgVTqu9v0SzwBe3bW 
> >> ClT+KIJPQ0LUqF/JQMEEvX1h5e9Rg+/MIdCkgA68MJKvJWmkQFUNQhXDv4lnCVgx 
> >> pzPxx6KnA3likvbs24CO6dZ22zZVDIC73vXYmxaF/VOrem8Lau+rRhnsHCCs5uJw 
> >> +4KWX/AgVO9k64trEF0QurINtmpAuqSFNIbRTHsIU8a/dp+06z4oC3ORmiBjjQFR 
> >> ik4bGDVwaKwC/8+4Wr5ybk48wi30lwGRXaAoVH/he+RvZt6gLBx3agsR+UIEo0rI 
> >> NseeggzKs176ntp5wmWSRoaDMevj5xnBIpmTDsqvgu7wOSRGL77nDab/CAdIuMIJ 
> >> U5BzrF0w280FxxZgx/iVYRl4JeZstfHQlyBngvO0FgJosOX5UDN13txuMgsRPMoW 
> >> eFzkWjFdUe77WW/kv8BWrykRztcrLoAaiPS/b6R9LMGywcfKyC1m2wP0u4aAiCfa 
> >> y5TyxwpANFKXEWIfAyDraRcDDCI/B6ujG6p1r7RUNxzAB75fTh+hdXLqi1dvNtoQ 
> >> MFZPaeHEk+0+tlRTudnb 
> >> =Qp3b 
> >> -----END PGP SIGNATURE----- 
> >> 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "zodb" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to [email protected] <javascript:>. 
> > For more options, visit https://groups.google.com/d/optout. 
>
>
>
> -- 
> Jim Fulton 
> http://jimfulton.info 
>

-- 
You received this message because you are subscribed to the Google Groups "zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.