Connection Pools Increasing
Justin Dunsworth <[email protected]> Fri, 10 Aug 2018 10:06:05 -0500
| Newsgroups | gmane.comp.web.zope.general |
|---|---|
| Message-ID | <CALkJ9bF5C9Ddcp93eR9HMGh34wjUkzLWexocu_7Bv=iV2i1itA@mail.gmail.com> |
--===============6182865783582725996== Content-Type: multipart/alternative; boundary="000000000000cef7290573161571" --000000000000cef7290573161571 Content-Type: text/plain; charset="UTF-8" I've recently had an issue where my site becomes intermittently unavailable after years of working fine and I'm uncertain on how to troubleshoot where it's coming from. My Zope site is accessed through Apache via proxy. I randomly started receiving Apaches Unavailable page as it is unable to pass off the request. After looking through both Apaches logs and Zopes logs I see that Zope is throwing a ton of ZODB warnings and criticals relating to pool_size and open connections. 2018-08-09T06:17:46 WARNING ZODB.DB DB.open() has 9 open connections with a pool_size of 7 ------ 2018-08-09T06:22:01 CRITICAL ZODB.DB DB.open() has 293 open connections with a pool_size of 7 ------ 2018-08-09T06:23:00 CRITICAL ZODB.DB DB.open() has 294 open connections with a pool_size of 7 ------ 2018-08-09T06:23:32 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size of 7 ------ 2018-08-09T06:23:57 CRITICAL ZODB.DB DB.open() has 295 open connections with a pool_size of 7 ------ 2018-08-09T06:29:41 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size of 7 ------ 2018-08-09T06:29:41 WARNING ZODB.DB DB.open() has 9 open connections with a pool_size of 7 ------ 2018-08-09T06:34:10 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size of 7 ------ 2018-08-09T06:37:16 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size of 7 ------ 2018-08-09T06:37:16 WARNING ZODB.DB DB.open() has 9 open connections with a pool_size of 7 ------ 2018-08-09T06:41:04 CRITICAL ZODB.DB DB.open() has 296 open connections with a pool_size of 7 ------ 2018-08-09T06:44:27 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size of 7 ------ 2018-08-09T06:44:41 CRITICAL ZODB.DB DB.open() has 297 open connections with a pool_size of 7 ------ 2018-08-09T06:47:13 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size of 7 ------ 2018-08-09T06:47:13 WARNING ZODB.DB DB.open() has 9 open connections with a pool_size of 7 ------ 2018-08-09T06:47:14 WARNING ZODB.DB DB.open() has 10 open connections with a pool_size of 7 During this, Apache is throwing the following errors: [Thu Aug 09 08:39:55 2018] [error] (OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted. : proxy: HTTP: attempt to connect to 127.0.0.1:1337 (*) failed [Thu Aug 09 08:41:00 2018] [error] (OS 10048)Only one usage of each socket address (protocol/network address/port) is normally permitted. : proxy: HTTP: attempt to connect to 127.0.0.1:1337 (*) failed [Thu Aug 09 08:41:38 2018] [error] (OS 10061)No connection could be made because the target machine actively refused it. : proxy: HTTP: attempt to connect to 127.0.0.1:1337 (*) failed [Thu Aug 09 08:41:39 2018] [error] (OS 10061)No connection could be made because the target machine actively refused it. : proxy: HTTP: attempt to connect to 127.0.0.1:1337 (*) failed I'm thinking that because of the large amount of open connections Zope is no longer serving requests and Apache is throwing the unavailable upon proxy. My problem is that I'm not quite sure how to figure out what's using the amount of connections and how to get them to release. Any help would be appreciated. Thanks! --000000000000cef7290573161571 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>I've recently had an issue where my site becomes = intermittently unavailable after years of working fine and I'm uncertai= n on how to troubleshoot where it's coming from.<br></div><div><br></di= v><div>My Zope site is accessed through Apache via proxy. I randomly starte= d receiving Apaches Unavailable page as it is unable to pass off the reques= t. After looking through both Apaches logs and Zopes logs I see that Zope i= s throwing a ton of ZODB warnings and criticals relating to pool_size and o= pen connections.=C2=A0</div><div><br></div><div><div>2018-08-09T06:17:46 WA= RNING ZODB.DB DB.open() has 9 open connections with a pool_size of 7</div><= div>------</div><div>2018-08-09T06:22:01 CRITICAL ZODB.DB DB.open() has 293= open connections with a pool_size of 7</div><div>------</div><div>2018-08-= 09T06:23:00 CRITICAL ZODB.DB DB.open() has 294 open connections with a pool= _size of 7</div><div>------</div><div>2018-08-09T06:23:32 WARNING ZODB.DB D= B.open() has 8 open connections with a pool_size of 7</div><div>------</div= ><div>2018-08-09T06:23:57 CRITICAL ZODB.DB DB.open() has 295 open connectio= ns with a pool_size of 7</div><div>------</div><div>2018-08-09T06:29:41 WAR= NING ZODB.DB DB.open() has 8 open connections with a pool_size of 7</div><d= iv>------</div><div>2018-08-09T06:29:41 WARNING ZODB.DB DB.open() has 9 ope= n connections with a pool_size of 7</div><div>------</div><div>2018-08-09T0= 6:34:10 WARNING ZODB.DB DB.open() has 8 open connections with a pool_size o= f 7</div><div>------</div><div>2018-08-09T06:37:16 WARNING ZODB.DB DB.open(= ) has 8 open connections with a pool_size of 7</div><div>------</div><div>2= 018-08-09T06:37:16 WARNING ZODB.DB DB.open() has 9 open connections with a = pool_size of 7</div><div>------</div><div>2018-08-09T06:41:04 CRITICAL ZODB= .DB DB.open() has 296 open connections with a pool_size of 7</div><div>----= --</div><div>2018-08-09T06:44:27 WARNING ZODB.DB DB.open() has 8 open conne= ctions with a pool_size of 7</div><div>------</div><div>2018-08-09T06:44:41= CRITICAL ZODB.DB DB.open() has 297 open connections with a pool_size of 7<= /div><div>------</div><div>2018-08-09T06:47:13 WARNING ZODB.DB DB.open() ha= s 8 open connections with a pool_size of 7</div><div>------</div><div>2018-= 08-09T06:47:13 WARNING ZODB.DB DB.open() has 9 open connections with a pool= _size of 7</div><div>------</div><div>2018-08-09T06:47:14 WARNING ZODB.DB D= B.open() has 10 open connections with a pool_size of 7</div></div><div><br>= </div><div>During this, Apache is throwing the following errors:=C2=A0</div= ><div><div>[Thu Aug 09 08:39:55 2018] [error] (OS 10048)Only one usage of e= ach socket address (protocol/network address/port) is normally permitted.= =C2=A0 : proxy: HTTP: attempt to connect to=C2=A0<a href=3D"http://127.0.0.= 1:1337/" target=3D"_blank">127.0.0.1:1337</a>=C2=A0(*) failed</div><div>[Th= u Aug 09 08:41:00 2018] [error] (OS 10048)Only one usage of each socket add= ress (protocol/network address/port) is normally permitted.=C2=A0 : proxy: = HTTP: attempt to connect to=C2=A0<a href=3D"http://127.0.0.1:1337/" target= =3D"_blank">127.0.0.1:1337</a>=C2=A0(*) failed</div><div>[Thu Aug 09 08:41:= 38 2018] [error] (OS 10061)No connection could be made because the target m= achine actively refused it.=C2=A0 : proxy: HTTP: attempt to connect to=C2= =A0<a href=3D"http://127.0.0.1:1337/" target=3D"_blank">127.0.0.1:1337</a>= =C2=A0(*) failed</div><div>[Thu Aug 09 08:41:39 2018] [error] (OS 10061)No = connection could be made because the target machine actively refused it.=C2= =A0 : proxy: HTTP: attempt to connect to=C2=A0<a href=3D"http://127.0.0.1:1= 337/" target=3D"_blank">127.0.0.1:1337</a>=C2=A0(*) failed</div></div><div>= <br></div><div>I'm thinking that because of the large amount of open co= nnections Zope is no longer serving requests and Apache is throwing the una= vailable upon proxy.=C2=A0<br></div><div><br></div><div>My problem is that = I'm not quite sure how to figure out what's using the amount of con= nections and how to get them to release. Any help would be appreciated.</di= v><div><br></div><div>Thanks!<div class=3D"gmail-yj6qo gmail-ajU" style=3D"= outline:none;padding:10px 0px;width:22px;margin:2px 0px 0px"><br class=3D"g= mail-Apple-interchange-newline"></div></div></div> --000000000000cef7290573161571-- --===============6182865783582725996== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Zope maillist - [email protected] https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) --===============6182865783582725996==--