Re: stunnel with OpenSSL engine: reload leaks HSM connections
Brent Kimberley <[email protected]> Fri, 27 Sep 2019 19:54:14 +0000 (UTC)
| Newsgroups | gmane.network.stunnel.user |
|---|---|
| Message-ID | <[email protected]> |
--===============4203079229420713399==
Content-Type: multipart/alternative;
boundary="----=_Part_270224_253031454.1569614054151"
------=_Part_270224_253031454.1569614054151
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Sounds slightly related to "After generating a new key-pair to HSM,=C2=A0E=
NGINE_load_private_key()=C2=A0still returns the old key."https://stackoverf=
low.com/questions/40676573/how-to-reload-key-from-hsm-by-using-openssl
Suggestions at the time included calling ENGINE_cleanup(), ENGINE_finish() =
and ENGINE_init()=C2=A0They cleaned-up the engine and then re-opened openss=
l/crypto.=C2=A0 =C2=A0It was slow and affected TLS connections using other =
key-pairs
Consider running one service per port, or better yet, one service per port-=
connection.
On Friday, September 27, 2019, 12:57:48 p.m. EDT, Lynch, Andrew <andrew=
[email protected]> wrote: =20
=20
<!--#yiv5227285775 _filtered #yiv5227285775 {font-family:Helvetica;panose=
-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv5227285775 {font-family:Helvetica;p=
anose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv5227285775 {font-family:Calibr=
i;panose-1:2 15 5 2 2 2 4 3 2 4;} _filtered #yiv5227285775 {font-family:Tah=
oma;panose-1:2 11 6 4 3 5 4 4 2 4;}#yiv5227285775 #yiv5227285775 p.yiv52272=
85775MsoNormal, #yiv5227285775 li.yiv5227285775MsoNormal, #yiv5227285775 di=
v.yiv5227285775MsoNormal {margin:0cm;margin-bottom:.0001pt;font-size:12.0pt=
;font-family:"Times New Roman", "serif";}#yiv5227285775 a:link, #yiv5227285=
775 span.yiv5227285775MsoHyperlink {color:blue;text-decoration:underline;}#=
yiv5227285775 a:visited, #yiv5227285775 span.yiv5227285775MsoHyperlinkFollo=
wed {color:purple;text-decoration:underline;}#yiv5227285775 span.yiv5227285=
775E-MailFormatvorlage17 {font-family:"Calibri", "sans-serif";color:#1F497D=
;}#yiv5227285775 .yiv5227285775MsoChpDefault {font-size:10.0pt;} _filtered =
#yiv5227285775 {margin:70.85pt 70.85pt 2.0cm 70.85pt;}#yiv5227285775 div.yi=
v5227285775WordSection1 {}-->
I=E2=80=99ve been digging around the code and I am no longer sure whether s=
tunnel is at fault or it is actually an issue in OpenSSL!
=20
=C2=A0
=20
load_key_engine() in ctx.c shoves the key into the SSL context =E2=80=93 SS=
L_CTX_use_PrivateKey(section->ctx, pkey).=C2=A0 So the key should be freed =
when that context is disposed of.
=20
=C2=A0
=20
The SIGHUP ends up in case SIGNAL_RELOAD_CONFIG of signal_pipe_dispatch(), =
which calls options_free()=C2=A0 [stunnel.c line 830].
=20
=C2=A0
=20
Eventually there is an SSL_CTX_free() call, but it looks like this does not=
actually free the private key.=C2=A0 Ephemeral keys are freed and loads of=
other stuff, but not the private key associated with the certificate.
=20
=C2=A0
=20
I may try to build a hacked OpenSSL on Monday to see what happens if SSL_CT=
X_free also frees the private key=E2=80=A6
=20
=C2=A0
=20
Regards,
=20
Andrew.
=20
| | |
=C2=A0
=20
=C2=A0
=20
From: Brent Kimberley [mailto:[email protected]]
Sent: Friday, September 27, 2019 4:29 PM
To: [email protected]; Lynch, Andrew
Subject: Re: stunnel-users Digest, Vol 182, Issue 14
=20
=C2=A0
=20
>> Over time, after numerous reloads of stunnel (kill -HUP) the HSM reports=
that its connection table is full.=C2=A0
=20
>>=C2=A0 Logging from the engine shows that stunnel is never freeing the ke=
ys and therefore the engine is not closing the associated sessions with the=
HSM.=C2=A0 Each stunnel reload opens 70 new sessions until eventually the =
HSM's configured limit is exceeded.
=20
=C2=A0
=20
Where does the signal dispatcher free keys on SIG_HUP?
=20
https://github.com/mtrojnar/stunnel/blob/master/src/stunnel.c
=20
=C2=A0
=20
=C2=A0
=20
Date: Fri, 27 Sep 2019 07:23:29 +0000
=20
From: "Lynch, Andrew" <[email protected]>
=20
To: Eric Eberhard <[email protected]>
=20
Cc: "[email protected]" <[email protected]>
=20
Subject: Re: [stunnel-users] stunnel with OpenSSL engine: reload leaks HSM =
connections
=20
=C2=A0
=20
Hi Eric,
=20
=C2=A0
=20
Thank you for your suggestion.=C2=A0 It is certainly worth looking into, al=
though I suspect it may be impractical in our environment.
=20
=C2=A0
=20
Now we have a single stunnel.conf with 70 services which would translate in=
to 70 additional entries in inetd.conf.=C2=A0 Each of those entries then ne=
eds to have its own reduced stunnel.conf so that it will only load the cert=
and key for its endpoint.
=20
=C2=A0
=20
The stunnel HOWTO recommends daemon mode over inetd mode due to the overhea=
d for forking and SSL initialization.=C2=A0 We would have additional overhe=
ad for engine init and HSM connection.
=20
=C2=A0
=20
Regards,
=20
Andrew.
=20
=C2=A0
=20
-----Original Message-----
=20
From: Eric Eberhard [mailto:[email protected]]
=20
Sent: Thursday, September 26, 2019 7:20 PM
=20
To: Lynch, Andrew;[email protected]
=20
Subject: RE: [stunnel-users] stunnel with OpenSSL engine: reload leaks HSM =
connections?
=20
=C2=A0
=20
Try running in inetd mode -- even if you don't like this you will learn som=
ething.=C2=A0 Inetd will close connections as needed.=C2=A0 E
=20
=C2=A0
=20
-----Original Message-----
=20
From: stunnel-users [mailto:[email protected]] On Behalf Of=
Lynch, Andrew
=20
Sent: Thursday, September 26, 2019 5:56 AM
=20
To:[email protected]
=20
Subject: [stunnel-users] stunnel with OpenSSL engine: reload leaks HSM conn=
ections?
=20
=C2=A0
=20
Hi,
=20
=C2=A0
=20
We are using stunnel as a server to terminate incoming TLS connections.=C2=
=A0 The config has around 70 services with certificates whose EC private ke=
ys are stored in an HSM and accessed using an OpenSSL engine.
=20
=C2=A0
=20
Over time, after numerous reloads of stunnel (kill -HUP) the HSM reports th=
at its connection table is full.=C2=A0 Logging from the engine shows that s=
tunnel is never freeing the keys and therefore the engine is not closing th=
e associated sessions with the HSM.=C2=A0 Each stunnel reload opens 70 new =
sessions until eventually the HSM's configured limit is exceeded.
=20
=C2=A0
=20
This behaviour has been observed on Suse Enterprise Linux 12.3 with the sys=
tem-provided stunnel-5.00-4.3.4, but I can reproduce it with my own build o=
f the current version 5.55.
=20
=C2=A0
=20
Is this a known issue?=C2=A0 It appears that other (ephemeral) keys are bei=
ng freed, just not those associated with the service certificates.
=20
=C2=A0
=20
Currently our workaround is to perform a full restart instead of a reload.
=20
This closes all HSM sessions when the process terminates, but of course it =
also kills any open client connections so it can only be done during the sc=
heduled maintenance windows.
=20
=C2=A0
=20
Regards,
=20
Andrew.
=20
=C2=A0
=20
_______________________________________________
=20
stunnel-users mailing list
=20
[email protected]
=20
https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users
=20
=C2=A0
=20
=C2=A0
=20
=C2=A0
=20
=C2=A0
=20
------------------------------
=20
=C2=A0
=20
Subject: Digest Footer
=20
=C2=A0
=20
_______________________________________________
=20
stunnel-users mailing list
=20
[email protected]
=20
https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users
=20
=C2=A0
=20
=C2=A0
=20
------------------------------
=20
=C2=A0
=20
End of stunnel-users Digest, Vol 182, Issue 14
=20
**********************************************
=20
------=_Part_270224_253031454.1569614054151
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<html><head></head><body><div class=3D"ydp25dd2242yahoo-style-wrap" style=
=3D"font-family:Helvetica Neue, Helvetica, Arial, sans-serif;font-size:16px=
;"><div></div>
<div dir=3D"ltr" data-setdir=3D"false">Sounds slightly related to "=
<span><span style=3D"color: rgb(36, 39, 41); font-family: Arial, Helvetica =
Neue, Helvetica, sans-serif; font-size: 15px;">After generating a new key-p=
air to HSM, </span><code style=3D"margin: 0px; padding: 1px 5px; borde=
r: 0px; font-stretch: inherit; line-height: inherit; font-family: Consolas,=
Menlo, Monaco, Lucida Console, Liberation Mono, DejaVu Sans Mono, Bitstrea=
m Vera Sans Mono, Courier New, monospace, sans-serif; vertical-align: basel=
ine; background-color: rgb(239, 240, 241); white-space: pre-wrap; color: rg=
b(36, 39, 41);">ENGINE_load_private_key()</code><span style=3D"color: rgb(3=
6, 39, 41); font-family: Arial, Helvetica Neue, Helvetica, sans-serif; font=
-size: 15px;"> still returns the old key."</span></span></div><div dir=
=3D"ltr" data-setdir=3D"false"><a href=3D"https://stackoverflow.com/questio=
ns/40676573/how-to-reload-key-from-hsm-by-using-openssl" rel=3D"nofollow" t=
arget=3D"_blank">https://stackoverflow.com/questions/40676573/how-to-reload=
-key-from-hsm-by-using-openssl</a><br></div><div dir=3D"ltr" data-setdir=3D=
"false"><span><span style=3D"color: rgb(36, 39, 41); font-family: Arial, He=
lvetica Neue, Helvetica, sans-serif; font-size: 15px;"><br></span></span></=
div><div dir=3D"ltr" data-setdir=3D"false"><span><span style=3D"color: rgb(=
36, 39, 41); font-family: Arial, Helvetica Neue, Helvetica, sans-serif; fon=
t-size: 15px;"><span><span style=3D"color: rgb(36, 39, 41); font-family: Ar=
ial, Helvetica Neue, Helvetica, sans-serif;">Suggestions at the time includ=
ed calling </span></span></span></span><span style=3D"color: rgb(36, 39, 41=
); font-family: Arial, Helvetica Neue, Helvetica, sans-serif; font-size: 15=
px;">ENGINE_cleanup(), ENGINE_finish() and ENGINE_init() </span></div>=
<div dir=3D"ltr" data-setdir=3D"false"><span style=3D"color: rgb(36, 39, 41=
); font-family: Arial, Helvetica Neue, Helvetica, sans-serif; font-size: 15=
px;">They cleaned-up the engine and then re-opened openssl/crypto. &n=
bsp;</span><span style=3D"color: rgb(36, 39, 41); font-family: Arial, Helve=
tica Neue, Helvetica, sans-serif; font-size: 15px;">It was slow and affecte=
d TLS connections using other key-pairs</span></div><div dir=3D"ltr" data-s=
etdir=3D"false"><br></div><div dir=3D"ltr" data-setdir=3D"false">Consider r=
unning one service per port, or better yet, one service per port-connection=
.</div><div dir=3D"ltr" data-setdir=3D"false"><br><div><br></div><div><br><=
/div><div><br></div><br></div><div><br></div>
=20
</div><div id=3D"yahoo_quoted_0512947318" class=3D"yahoo_quoted">
<div style=3D"font-family:'Helvetica Neue', Helvetica, Arial, s=
ans-serif;font-size:13px;color:#26282a;">
=20
<div>
On Friday, September 27, 2019, 12:57:48 p.m. EDT, Lynch=
, Andrew <[email protected]> wrote:
</div>
<div><br></div>
<div><br></div>
<div><div id=3D"yiv5227285775">
=20
=20
<style><!--
#yiv5227285775 =20
_filtered #yiv5227285775 {font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
_filtered #yiv5227285775 {font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
_filtered #yiv5227285775 {font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
_filtered #yiv5227285775 {font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
#yiv5227285775 =20
#yiv5227285775 p.yiv5227285775MsoNormal, #yiv5227285775 li.yiv5227285775Mso=
Normal, #yiv5227285775 div.yiv5227285775MsoNormal
=09{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman", "serif";}
#yiv5227285775 a:link, #yiv5227285775 span.yiv5227285775MsoHyperlink
=09{
color:blue;
text-decoration:underline;}
#yiv5227285775 a:visited, #yiv5227285775 span.yiv5227285775MsoHyperlinkFoll=
owed
=09{
color:purple;
text-decoration:underline;}
#yiv5227285775 span.yiv5227285775E-MailFormatvorlage17
=09{
font-family:"Calibri", "sans-serif";
color:#1F497D;}
#yiv5227285775 .yiv5227285775MsoChpDefault
=09{
font-size:10.0pt;}
_filtered #yiv5227285775 {
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
#yiv5227285775 div.yiv5227285775WordSection1
=09{}
--></style>
<div>
<div class=3D"yiv5227285775WordSection1">
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">I=E2=80=99ve been digging around the code and I am =
no longer sure whether stunnel is at fault or it is actually an issue in Op=
enSSL!</span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;"> </span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">load_key_engine() in ctx.c shoves the key into the =
SSL context =E2=80=93 SSL_CTX_use_PrivateKey(section->ctx, pkey). =
So the key should be freed when that
context is disposed of.</span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;"> </span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">The SIGHUP ends up in case SIGNAL_RELOAD_CONFIG of =
signal_pipe_dispatch(), which calls options_free() [stunnel.c line 83=
0].</span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;"> </span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">Eventually there is an SSL_CTX_free() call, but it =
looks like this does not actually free the private key. Ephemeral key=
s are freed and loads
of other stuff, but not the private key associated with the certificate.</=
span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;"> </span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">I may try to build a hacked OpenSSL on Monday to se=
e what happens if SSL_CTX_free also frees the private key=E2=80=A6</span></=
p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;"> </span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">Regards,</span></p>=20
<p class=3D"yiv5227285775MsoNormal"><span lang=3D"EN-US" style=3D"font-size=
:11.0pt;color:#1F497D;">Andrew.</span></p>=20
<div>
<table class=3D"yiv5227285775MsoNormalTable" border=3D"0" cellspacing=3D"0"=
cellpadding=3D"0" width=3D"100%" style=3D"width:100.0%;border-collapse:col=
lapse;">
<tbody>
<tr>
<td width=3D"20" style=3D"width:15.3pt;padding:0cm 15.0pt 0cm 0cm;"></td>
<td width=3D"1177" style=3D"width:882.45pt;border:none;border-left:solid gr=
ay 1.5pt;padding:0cm 0cm 0cm 18.75pt;">
</td>
</tr>
</tbody>
</table>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:11.0pt;color:#=
1F497D;"> </span></p>=20
</div>
<p class=3D"yiv5227285775MsoNormal"> </p>=20
<div>
<div style=3D"border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm =
0cm 0cm;">
<p class=3D"yiv5227285775MsoNormal"><b><span style=3D"font-size:10.0pt;">Fr=
om:</span></b><span style=3D"font-size:10.0pt;"> Brent Kimberley [mailto:br=
[email protected]]
<br>
<b>Sent:</b> Friday, September 27, 2019 4:29 PM<br>
<b>To:</b> [email protected]; Lynch, Andrew<br>
<b>Subject:</b> Re: stunnel-users Digest, Vol 182, Issue 14</span></p>=20
</div>
</div>
<p class=3D"yiv5227285775MsoNormal"> </p>=20
<div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"">>> <span style=
=3D"color:#26282A;">
Over time, after numerous reloads of stunnel (kill -HUP) the HSM reports th=
at its connection table is full. </span></span></p>=20
</div>
<div>
<div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"color:#26282A;">>>=
Logging from the engine shows that stunnel is never freeing the keys=
and therefore the engine is not closing the associated sessions with the H=
SM. Each stunnel reload
opens 70 new sessions until eventually the HSM's configured limit is excee=
ded.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"color:#26282A;"> <=
/span></p>=20
</div>
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"color:#26282A;">Where do=
es the signal dispatcher free keys on SIG_HUP?</span></p>=20
</div>
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D""><a rel=3D"nofollow" ta=
rget=3D"_blank" href=3D"https://github.com/mtrojnar/stunnel/blob/master/src=
/stunnel.c">https://github.com/mtrojnar/stunnel/blob/master/src/stunnel.c</=
a></span></p>=20
</div>
<div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D""> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D""> </span></p>=20
</div>
</div>
</div>
<div id=3D"yiv5227285775ydpaf216824yahoo_quoted_0356605184">
<div>
<div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Date: Fri, 27 Sep 2019 07:23:29 +0000</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">From: "Lynch, Andrew" <<a rel=3D"nofollow" ymailto=3D"mailto:an=
[email protected]" target=3D"_blank" href=3D"mailto:[email protected]=
">[email protected]</a>></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">To: Eric Eberhard <<a rel=3D"nofollow" ymailto=3D"mailto:flash@=
vicsmba.com" target=3D"_blank" href=3D"mailto:[email protected]">flash@vics=
mba.com</a>></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Cc: "<a rel=3D"nofollow" ymailto=3D"mailto:[email protected]=
rg" target=3D"_blank" href=3D"mailto:[email protected]">stunnel-use=
[email protected]</a>" <<a rel=3D"nofollow" ymailto=3D"mailto:stunnel-users=
@stunnel.org" target=3D"_blank" href=3D"mailto:[email protected]">s=
[email protected]</a>></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Subject: Re: [stunnel-users] stunnel with OpenSSL engine: reload l=
eaks HSM connections</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Hi Eric,</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Thank you for your suggestion. It is certainly worth looking=
into, although I suspect it may be impractical in our environment.</span><=
/p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Now we have a single stunnel.conf with 70 services which would tra=
nslate into 70 additional entries in inetd.conf. Each of those entrie=
s then needs to have
its own reduced stunnel.conf so that it will only load the cert and key fo=
r its endpoint.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">The stunnel HOWTO recommends daemon mode over inetd mode due to th=
e overhead for forking and SSL initialization. We would have addition=
al overhead for engine
init and HSM connection.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Regards,</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Andrew.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">-----Original Message-----</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">From: Eric Eberhard [mailto:<a rel=3D"nofollow" ymailto=3D"mailto:=
[email protected]" target=3D"_blank" href=3D"mailto:[email protected]">flas=
[email protected]</a>]
</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Sent: Thursday, September 26, 2019 7:20 PM</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">To: Lynch, Andrew;
<a rel=3D"nofollow" ymailto=3D"mailto:[email protected]" target=3D"=
_blank" href=3D"mailto:[email protected]">[email protected]=
</a></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Subject: RE: [stunnel-users] stunnel with OpenSSL engine: reload l=
eaks HSM connections?</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Try running in inetd mode -- even if you don't like this you will =
learn something. Inetd will close connections as needed. E</spa=
n></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">-----Original Message-----</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">From: stunnel-users [mailto:<a rel=3D"nofollow" ymailto=3D"mailto:=
[email protected]" target=3D"_blank" href=3D"mailto:stunnel=
[email protected]">[email protected]</a>] On Behal=
f Of Lynch,
Andrew</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Sent: Thursday, September 26, 2019 5:56 AM</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">To:
<a rel=3D"nofollow" ymailto=3D"mailto:[email protected]" target=3D"=
_blank" href=3D"mailto:[email protected]">[email protected]=
</a></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Subject: [stunnel-users] stunnel with OpenSSL engine: reload leaks=
HSM connections?</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Hi,</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">We are using stunnel as a server to terminate incoming TLS connect=
ions. The config has around 70 services with certificates whose EC pr=
ivate keys are stored
in an HSM and accessed using an OpenSSL engine.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Over time, after numerous reloads of stunnel (kill -HUP) the HSM r=
eports that its connection table is full. Logging from the engine sho=
ws that stunnel is
never freeing the keys and therefore the engine is not closing the associa=
ted sessions with the HSM. Each stunnel reload opens 70 new sessions =
until eventually the HSM's configured limit is exceeded.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">This behaviour has been observed on Suse Enterprise Linux 12.3 wit=
h the system-provided stunnel-5.00-4.3.4, but I can reproduce it with my ow=
n build of the
current version 5.55.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Is this a known issue? It appears that other (ephemeral) key=
s are being freed, just not those associated with the service certificates.=
</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Currently our workaround is to perform a full restart instead of a=
reload.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">This closes all HSM sessions when the process terminates, but of c=
ourse it also kills any open client connections so it can only be done duri=
ng the scheduled
maintenance windows.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Regards,</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Andrew.</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">_______________________________________________</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">stunnel-users mailing list</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"><a rel=3D"nofollow" ymailto=3D"mailto:[email protected]" t=
arget=3D"_blank" href=3D"mailto:[email protected]">stunnel-users@st=
unnel.org</a></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"><a rel=3D"nofollow" target=3D"_blank" href=3D"https://www.stunnel.=
org/cgi-bin/mailman/listinfo/stunnel-users">https://www.stunnel.org/cgi-bin=
/mailman/listinfo/stunnel-users</a></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">------------------------------</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">Subject: Digest Footer</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">_______________________________________________</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">stunnel-users mailing list</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"><a rel=3D"nofollow" ymailto=3D"mailto:[email protected]" t=
arget=3D"_blank" href=3D"mailto:[email protected]">stunnel-users@st=
unnel.org</a></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"><a rel=3D"nofollow" target=3D"_blank" href=3D"https://www.stunnel.=
org/cgi-bin/mailman/listinfo/stunnel-users">https://www.stunnel.org/cgi-bin=
/mailman/listinfo/stunnel-users</a></span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">------------------------------</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;"> </span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">End of stunnel-users Digest, Vol 182, Issue 14</span></p>=20
</div>
<div>
<p class=3D"yiv5227285775MsoNormal"><span style=3D"font-size:10.0pt;color:#=
26282A;">**********************************************</span></p>=20
</div>
</div>
</div>
</div>
</div>
</div>
</div></div>
</div>
</div></body></html>
------=_Part_270224_253031454.1569614054151--
--===============4203079229420713399==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
stunnel-users mailing list
[email protected]
https://www.stunnel.org/cgi-bin/mailman/listinfo/stunnel-users
--===============4203079229420713399==--