Re: stunnel-users Digest, Vol 182, Issue 14
Brent Kimberley <[email protected]> Fri, 27 Sep 2019 14:29:29 +0000 (UTC)
| Newsgroups | gmane.network.stunnel.user |
|---|---|
| Message-ID | <[email protected]> |
--===============8493072448690640294== Content-Type: multipart/alternative; boundary="----=_Part_153224_1712951411.1569594569010" ------=_Part_153224_1712951411.1569594569010 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable >> Over time, after numerous reloads of stunnel (kill -HUP) the HSM report= s that its connection table is full.=C2=A0>>=C2=A0 Logging from the engine = shows that stunnel is never freeing the keys and therefore the engine is no= t closing the associated sessions with the HSM.=C2=A0 Each stunnel reload o= pens 70 new sessions until eventually the HSM's configured limit is exceede= d. Where does the signal dispatcher free keys on SIG_HUP?https://github.com/mt= rojnar/stunnel/blob/master/src/stunnel.c Date: Fri, 27 Sep 2019 07:23:29 +0000 From: "Lynch, Andrew" <[email protected]> To: Eric Eberhard <[email protected]> Cc: "[email protected]" <[email protected]> Subject: Re: [stunnel-users] stunnel with OpenSSL engine: reload leaks HSM = connections Hi Eric, Thank you for your suggestion.=C2=A0 It is certainly worth looking into, al= though I suspect it may be impractical in our environment. Now we have a single stunnel.conf with 70 services which would translate in= to 70 additional entries in inetd.conf.=C2=A0 Each of those entries then ne= eds to have its own reduced stunnel.conf so that it will only load the cert= and key for its endpoint. The stunnel HOWTO recommends daemon mode over inetd mode due to the overhea= d for forking and SSL initialization.=C2=A0 We would have additional overhe= ad for engine init and HSM connection. Regards, Andrew. -----Original Message----- From: Eric Eberhard [mailto:[email protected]]=20 Sent: Thursday, September 26, 2019 7:20 PM To: Lynch, Andrew; [email protected] Subject: RE: [stunnel-users] stunnel with OpenSSL engine: reload leaks HSM = connections? Try running in inetd mode -- even if you don't like this you will learn som= ething.=C2=A0 Inetd will close connections as needed.=C2=A0 E -----Original Message----- From: stunnel-users [mailto:[email protected]] On Behalf Of= Lynch, Andrew Sent: Thursday, September 26, 2019 5:56 AM To: [email protected] Subject: [stunnel-users] stunnel with OpenSSL engine: reload leaks HSM conn= ections? Hi, We are using stunnel as a server to terminate incoming TLS connections.=C2= =A0 The config has around 70 services with certificates whose EC private ke= ys are stored in an HSM and accessed using an OpenSSL engine. Over time, after numerous reloads of stunnel (kill -HUP) the HSM reports th= at its connection table is full.=C2=A0 Logging from the engine shows that s= tunnel is never freeing the keys and therefore the engine is not closing th= e associated sessions with the HSM.=C2=A0 Each stunnel reload opens 70 new = sessions until eventually the HSM's configured limit is exceeded. This behaviour has been observed on Suse Enterprise Linux 12.3 with the sys= tem-provided stunnel-5.00-4.3.4, but I can reproduce it with my own build o= f the current version 5.55. Is this a known issue?=C2=A0 It appears that other (ephemeral) keys are bei= ng freed, just not those associated with the service certificates. Currently our workaround is to perform a full restart instead of a reload. This closes all HSM sessions when the process terminates, but of course it = also kills any open client connections so it can only be done during the sc= heduled maintenance windows. Regards, Andrew. _______________________________________________ stunnel-users mailing list [email protected] https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users ------------------------------ Subject: Digest Footer _______________________________________________ stunnel-users mailing list [email protected] https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users ------------------------------ End of stunnel-users Digest, Vol 182, Issue 14 ********************************************** =20 ------=_Part_153224_1712951411.1569594569010 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <html><head></head><body><div class=3D"ydpdad6c613yahoo-style-wrap" style= =3D"font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:16px= ;"><div></div> <div dir=3D"ltr" data-setdir=3D"false">>> <span><span style= =3D"color: rgb(38, 40, 42); font-family: Helvetica Neue, Helvetica, Arial, = sans-serif;">Over time, after numerous reloads of stunnel (kill -HUP) the H= SM reports that its connection table is full. </span></span></div><div= dir=3D"ltr" data-setdir=3D"false"><span><div><div dir=3D"ltr" style=3D"col= or: rgb(38, 40, 42); font-family: Helvetica Neue, Helvetica, Arial, sans-se= rif;">>> Logging from the engine shows that stunnel is never fr= eeing the keys and therefore the engine is not closing the associated sessi= ons with the HSM. Each stunnel reload opens 70 new sessions until eve= ntually the HSM's configured limit is exceeded.<br></div><div dir=3D"ltr" s= tyle=3D"color: rgb(38, 40, 42); font-family: Helvetica Neue, Helvetica, Ari= al, sans-serif;"><br></div></div><div dir=3D"ltr" style=3D"color: rgb(38, 4= 0, 42); font-family: Helvetica Neue, Helvetica, Arial, sans-serif;" data-se= tdir=3D"false">Where does the signal dispatcher free keys on SIG_HUP?</div>= </span></div><div dir=3D"ltr" data-setdir=3D"false"><a href=3D"https://gith= ub.com/mtrojnar/stunnel/blob/master/src/stunnel.c" rel=3D"nofollow" target= =3D"_blank">https://github.com/mtrojnar/stunnel/blob/master/src/stunnel.c</= a></div><div dir=3D"ltr" data-setdir=3D"false"><div><br></div><div><br></di= v></div></div><div id=3D"ydpaf216824yahoo_quoted_0356605184" class=3D"ydpaf= 216824yahoo_quoted"><div style=3D"font-family:'Helvetica Neue', Helvetica, = Arial, sans-serif;font-size:13px;color:#26282a;"><div><div dir=3D"ltr">Date= : Fri, 27 Sep 2019 07:23:29 +0000<br></div><div dir=3D"ltr">From: "Lynch, A= ndrew" <<a href=3D"mailto:[email protected]" rel=3D"nofollow" target= =3D"_blank">[email protected]</a>><br></div><div dir=3D"ltr">To: Eri= c Eberhard <<a href=3D"mailto:[email protected]" rel=3D"nofollow" target= =3D"_blank">[email protected]</a>><br></div><div dir=3D"ltr">Cc: "<a hre= f=3D"mailto:[email protected]" rel=3D"nofollow" target=3D"_blank">s= [email protected]</a>" <<a href=3D"mailto:[email protected]= rg" rel=3D"nofollow" target=3D"_blank">[email protected]</a>><br= ></div><div dir=3D"ltr">Subject: Re: [stunnel-users] stunnel with OpenSSL e= ngine: reload leaks HSM connections</div><div dir=3D"ltr"><br></div><div di= r=3D"ltr">Hi Eric,<br></div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Tha= nk you for your suggestion. It is certainly worth looking into, altho= ugh I suspect it may be impractical in our environment.<br></div><div dir= =3D"ltr"><br></div><div dir=3D"ltr">Now we have a single stunnel.conf with = 70 services which would translate into 70 additional entries in inetd.conf.= Each of those entries then needs to have its own reduced stunnel.con= f so that it will only load the cert and key for its endpoint.<br></div><di= v dir=3D"ltr"><br></div><div dir=3D"ltr">The stunnel HOWTO recommends daemo= n mode over inetd mode due to the overhead for forking and SSL initializati= on. We would have additional overhead for engine init and HSM connect= ion.<br></div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Regards,<br></div= ><div dir=3D"ltr">Andrew.<br></div><div dir=3D"ltr"><br></div><div dir=3D"l= tr">-----Original Message-----<br></div><div dir=3D"ltr">From: Eric Eberhar= d [mailto:<a href=3D"mailto:[email protected]" rel=3D"nofollow" target=3D"_= blank">[email protected]</a>] <br></div><div dir=3D"ltr">Sent: Thursday, Se= ptember 26, 2019 7:20 PM<br></div><div dir=3D"ltr">To: Lynch, Andrew; <a hr= ef=3D"mailto:[email protected]" rel=3D"nofollow" target=3D"_blank">= [email protected]</a><br></div><div dir=3D"ltr">Subject: RE: [stunn= el-users] stunnel with OpenSSL engine: reload leaks HSM connections?<br></d= iv><div dir=3D"ltr"><br></div><div dir=3D"ltr">Try running in inetd mode --= even if you don't like this you will learn something. Inetd will clo= se connections as needed. E<br></div><div dir=3D"ltr"><br></div><div = dir=3D"ltr">-----Original Message-----<br></div><div dir=3D"ltr">From: stun= nel-users [mailto:<a href=3D"mailto:[email protected]" rel= =3D"nofollow" target=3D"_blank">[email protected]</a>] On B= ehalf Of Lynch, Andrew<br></div><div dir=3D"ltr">Sent: Thursday, September = 26, 2019 5:56 AM<br></div><div dir=3D"ltr">To: <a href=3D"mailto:stunnel-us= [email protected]" rel=3D"nofollow" target=3D"_blank">[email protected]= rg</a><br></div><div dir=3D"ltr">Subject: [stunnel-users] stunnel with Open= SSL engine: reload leaks HSM connections?<br></div><div dir=3D"ltr"><br></d= iv><div dir=3D"ltr">Hi,<br></div><div dir=3D"ltr"><br></div><div dir=3D"ltr= ">We are using stunnel as a server to terminate incoming TLS connections.&n= bsp; The config has around 70 services with certificates whose EC private k= eys are stored in an HSM and accessed using an OpenSSL engine.<br></div><di= v dir=3D"ltr"><br></div><div dir=3D"ltr">Over time, after numerous reloads = of stunnel (kill -HUP) the HSM reports that its connection table is full.&n= bsp; Logging from the engine shows that stunnel is never freeing the keys a= nd therefore the engine is not closing the associated sessions with the HSM= . Each stunnel reload opens 70 new sessions until eventually the HSM'= s configured limit is exceeded.<br></div><div dir=3D"ltr"><br></div><div di= r=3D"ltr">This behaviour has been observed on Suse Enterprise Linux 12.3 wi= th the system-provided stunnel-5.00-4.3.4, but I can reproduce it with my o= wn build of the current version 5.55.<br></div><div dir=3D"ltr"><br></div><= div dir=3D"ltr">Is this a known issue? It appears that other (ephemer= al) keys are being freed, just not those associated with the service certif= icates.<br></div><div dir=3D"ltr"><br></div><div dir=3D"ltr">Currently our = workaround is to perform a full restart instead of a reload.<br></div><div = dir=3D"ltr">This closes all HSM sessions when the process terminates, but o= f course it also kills any open client connections so it can only be done d= uring the scheduled maintenance windows.<br></div><div dir=3D"ltr"><br></di= v><div dir=3D"ltr">Regards,<br></div><div dir=3D"ltr">Andrew.<br></div><div= dir=3D"ltr"><br></div><div dir=3D"ltr">___________________________________= ____________<br></div><div dir=3D"ltr">stunnel-users mailing list<br></div>= <div dir=3D"ltr"><a href=3D"mailto:[email protected]" rel=3D"nofoll= ow" target=3D"_blank">[email protected]</a><br></div><div dir=3D"lt= r"><a href=3D"https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-user= s" rel=3D"nofollow" target=3D"_blank">https://www.stunnel.org/cgi-bin/mailm= an/listinfo/stunnel-users</a><br></div><div dir=3D"ltr"><br></div><div dir= =3D"ltr"><br></div><div dir=3D"ltr"><br></div><div dir=3D"ltr"><br></div><d= iv dir=3D"ltr">------------------------------<br></div><div dir=3D"ltr"><br= ></div><div dir=3D"ltr">Subject: Digest Footer<br></div><div dir=3D"ltr"><b= r></div><div dir=3D"ltr">_______________________________________________<br= ></div><div dir=3D"ltr">stunnel-users mailing list<br></div><div dir=3D"ltr= "><a href=3D"mailto:[email protected]" rel=3D"nofollow" target=3D"_= blank">[email protected]</a><br></div><div dir=3D"ltr"><a href=3D"h= ttps://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users" rel=3D"nofol= low" target=3D"_blank">https://www.stunnel.org/cgi-bin/mailman/listinfo/stu= nnel-users</a><br></div><div dir=3D"ltr"><br></div><div dir=3D"ltr"><br></d= iv><div dir=3D"ltr">------------------------------<br></div><div dir=3D"ltr= "><br></div><div dir=3D"ltr">End of stunnel-users Digest, Vol 182, Issue 14= <br></div><div dir=3D"ltr">**********************************************<b= r></div></div> </div> </div></body></html> ------=_Part_153224_1712951411.1569594569010-- --===============8493072448690640294== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ stunnel-users mailing list [email protected] https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users --===============8493072448690640294==--