Re: SSL handshake crash
Ulf Wiger <[email protected]> Fri, 25 Dec 2015 21:40:09 +0100
| Newsgroups | gmane.comp.lang.erlang.bugs |
|---|---|
| Message-ID | <CADPRLo-YmQofubWL6KiGfnm3NBYcQamfgJG=hs3Nc+GTtBDSFg@mail.gmail.com> |
--===============6789705141003137531== Content-Type: multipart/alternative; boundary=001a113f8ec21cb6a90527bef613 --001a113f8ec21cb6a90527bef613 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi Ingela, 'active' should be set to false: https://github.com/PDXostc/rvi_core/blob/develop/components/dlink_tls/src/d= link_tls_conn.erl#L346 BR, Ulf W 2015-12-25 12:54 GMT+01:00 Ingela Anderton Andin < [email protected]>: > Hi! > > > > From ssl users guide > > > > "Ensure active is set to false before trying to upgrade a connection to > an SSL connection, otherwise SSL handshake messages can be delivered to t= he > wrong process." > > > > Regards Ingela Erlang/OTP team - Ericsson AB > ------------------------------ > *Fr=C3=A5n:* [email protected] [[email protected]= rg] > f=C3=B6r Danil Zagoskin [[email protected]] > *Skickat:* den 24 december 2015 10:22 > *Till:* Ulf Wiger > *Kopia:* [email protected] > *=C3=84mne:* Re: [erlang-bugs] SSL handshake crash > > Hi! > > I have the same issue, but not so often. > It seems to appear only when upgrading plain socket to TLS (XMPP starttls > in my case). > > Possibly it's some kind of race condition when client sends TLS hello > before server does ssl_accept(). Maybe some active/passive socket mode > issue. > > If you control the client code, could you add some sleep before starttls > and check if that fixes the issue? > > On Wed, Dec 23, 2015 at 8:38 PM, Ulf Wiger <[email protected]> wrote: > >> Hmm=E2=80=A6 I send this to erlang-bugs, but it didn=E2=80=99t seem to g= et through. >> >> When connecting some Android software to an Erlang node using TLS, we >> sometimes (about 1 in 3 or 4 times) get the following errors: >> >> 2015-12-22 15:31:00.772 [error] <0.210.0> gen_fsm <0.210.0> in state >> hello terminated with reason: no function clause matching >> ssl_handshake:update_handshake_history(undefined, <<1,0,0,175,3,1,86,121= ,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247,96,14,32,123,176,..= .>>) >> line 450 >> >> 15:31:00.783<dlink_tls_conn/327>dlink_tls_conn:terminate(): Reason: >> {{function_clause,[{ssl_handshake,update_handshake_history,[undefined,<<= 1,0,0,175,3,1,86,121,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247= ,96,14,32,123,176,109,210,170,150,204,23,32,228,0,0,70,0,4,0,5,0,47,0,53,19= 2,2,192,4,192,5,192,12,192,14,192,15,192,7,192,9,192,10,192,17,192,19,192,2= 0,0,51,0,57,0,50,0,56,0,10,192,3,192,13,192,8,192,18,0,22,0,19,0,9,0,21,0,1= 8,0,3,0,8,0,20,0,17,0,255,1,0,0,64,0,11,0,4,3,0,1,2,0,10,0,52,0,50,0,14,0,1= 3,0,25,0,11,0,12,0,24,0,9,0,10,0,22,0,23,0,8,0,6,0,7,0,20,0,21,0,4,0,5,0,18= ,0,19,0,1,0,2,0,3,0,15,0,16,0,17>>],[{file,"ssl_handshake.erl"},{line,450}]= },{tls_connection,'-next_state/4-fun-0-',3,[{file,"tls_connection.erl"},{li= ne,458}]},{tls_connection,next_state,4,[{file,"tls_connection.erl"},{line,4= 67}]},{gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,518}]},{proc_lib,in= it_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},{gen_fsm,sync_send_al= l_state_event,[<0.210.0>,{start,infinity},infinity]}} >> >> 2015-12-22 15:31:00.784 [error] <0.210.0> CRASH REPORT Process <0.210.0> >> with 0 neighbours exited with reason: no function clause matching >> ssl_handshake:update_handshake_history(undefined, <<1,0,0,175,3,1,86,121= ,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247,96,14,32,123,176,..= .>>) >> line 450 in gen_fsm:terminate/7 line 626 >> >> 2015-12-22 15:31:00.785 [error] <0.209.0> gen_server <0.209.0> terminate= d >> with reason: >> {{function_clause,[{ssl_handshake,update_handshake_history,[undefined,<<= 1,0,0,175,3,1,86,121,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247= ,96,14,32,123,176,109,210,170,150,204,23,32,228,0,0,70,0,4,0,5,0,47,0,53,19= 2,2,192,4,192,5,192,12,192,14,192,15,192,7,192,9,192,10,192,17,192,19,192,2= 0,0,51,0,57,0,50,0,56,0,10,192,3,192,13,192,8,192,18,0,22,0,19,0,9,0,21,0,1= 8,0,3,0,8,0,20,0,17,0,255,1,0,0,64,0,11,0,4,3,0,1,2,0,10,0,52,0,50,0,14,0,1= 3,0,25,0,11,0,12,0,24,0,9,0,10,0,22,0,23,0,8,0,...>>],...},...]},...} >> in gen_fsm:sync_send_all_state_event/3 line 257 >> >> 2015-12-22 15:31:00.786 [error] <0.209.0> CRASH REPORT Process <0.209.0> >> with 0 neighbours exited with reason: >> {{function_clause,[{ssl_handshake,update_handshake_history,[undefined,<<= 1,0,0,175,3,1,86,121,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247= ,96,14,32,123,176,109,210,170,150,204,23,32,228,0,0,70,0,4,0,5,0,47,0,53,19= 2,2,192,4,192,5,192,12,192,14,192,15,192,7,192,9,192,10,192,17,192,19,192,2= 0,0,51,0,57,0,50,0,56,0,10,192,3,192,13,192,8,192,18,0,22,0,19,0,9,0,21,0,1= 8,0,3,0,8,0,20,0,17,0,255,1,0,0,64,0,11,0,4,3,0,1,2,0,10,0,52,0,50,0,14,0,1= 3,0,25,0,11,0,12,0,24,0,9,0,10,0,22,0,23,0,8,0,...>>],...},...]},...} >> in gen_server:terminate/7 line 826 >> >> 2015-12-22 15:31:00.787 [error] <0.109.0> Supervisor tls_connection_sup >> had child undefined started with {tls_connection,start_link,undefined} a= t >> <0.210.0> exit with reason no function clause >> matching ssl_handshake:update_handshake_history(undefined, >> <<1,0,0,175,3,1,86,121,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,2= 9,247,96,14,32,123,176,...>>) >> line 450 in context child_terminated >> >> >> We run OTP Erlang/OTP 18 [erts-7.2] with ssl-7.2, and the erlang side ha= s >> the following options: >> >> [{verify,verify_peer}, >> {certfile,"/home/.../device_cert.crt=E2=80=9D}, >> {keyfile,"/home/.../device_key.pem=E2=80=9D}, >> {cacertfile,"/home/.../root_cert.crt=E2=80=9D}, >> {verify_fun,{#Fun<dlink_tls_conn.65.24728257>,{'RSAPublicKey=E2=80=99,..= .}}}, >> {partial_chain,#Fun<dlink_tls_conn.64.24728257>}] >> >> Basically, the verify_fun validates a self-signed cert >> >> https://github.com/PDXostc/rvi_core/blob/develop/components/dlink_tls/sr= c/dlink_tls_conn.erl#L393 >> >> and the partial_chain fun most likely does much less than it should >> >> https://github.com/PDXostc/rvi_core/blob/develop/components/dlink_tls/sr= c/dlink_tls_conn.erl#L421 >> >> On the Android side, we=E2=80=99re using Android 4.4.2 (API 19). >> >> It feels like a timing-related problem on the erlang side. >> >> Let me know if you need more information. >> >> BR, >> Ulf W >> >> _______________________________________________ >> erlang-bugs mailing list >> [email protected] >> http://erlang.org/mailman/listinfo/erlang-bugs >> >> > > > -- > Danil Zagoskin | [email protected] > --001a113f8ec21cb6a90527bef613 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Hi Ingela,<div><br></div><div>'active' should be s= et to false:</div><div><br></div><div><a href=3D"https://github.com/PDXostc= /rvi_core/blob/develop/components/dlink_tls/src/dlink_tls_conn.erl#L346">ht= tps://github.com/PDXostc/rvi_core/blob/develop/components/dlink_tls/src/dli= nk_tls_conn.erl#L346</a><br></div><div><br></div><div>BR,</div><div>Ulf W</= div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">2015-12= -25 12:54 GMT+01:00 Ingela Anderton Andin <span dir=3D"ltr"><<a href=3D"= mailto:[email protected]" target=3D"_blank">Ingela.Anderto= [email protected]</a>></span>:<br><blockquote class=3D"gmail_quote" s= tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div> <div style=3D"direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt= "> <p>Hi!</p> <p>=C2=A0</p> <p>From ssl users guide</p> <p>=C2=A0</p> <p>=C2=A0"Ensure active is set to false before trying to upgrade a con= nection to an SSL connection, otherwise SSL handshake messages can be deliv= ered to the wrong process."</p> <p>=C2=A0</p> <p>Regards Ingela Erlang/OTP team - Ericsson AB</p> <div style=3D"FONT-SIZE:16px;FONT-FAMILY:Times New Roman;COLOR:#000000"> <hr> <div style=3D"DIRECTION:ltr"><font color=3D"#000000" size=3D"2" face=3D"Tah= oma"><b>Fr=C3=A5n:</b> <a href=3D"mailto:[email protected]" ta= rget=3D"_blank">[email protected]</a> [<a href=3D"mailto:erlan= [email protected]" target=3D"_blank">[email protected]= </a>] f=C3=B6r Danil Zagoskin [<a href=3D"mailto:[email protected]" target=3D"_blan= k">[email protected]</a>]<br> <b>Skickat:</b> den 24 december 2015 10:22<br> <b>Till:</b> Ulf Wiger<br> <b>Kopia:</b> <a href=3D"mailto:[email protected]" target=3D"_blank">e= [email protected]</a><br> <b>=C3=84mne:</b> Re: [erlang-bugs] SSL handshake crash<br> </font><br> </div><div><div class=3D"h5"> <div></div> <div> <div dir=3D"ltr">Hi! <div><br> </div> <div>I have the same issue, but not so often.</div> <div>It seems to appear only when upgrading plain socket to TLS (XMPP start= tls in my case).</div> <div><br> </div> <div>Possibly it's some kind of race condition when client sends TLS he= llo before server does ssl_accept(). Maybe some active/passive socket mode = issue.</div> <div><br> </div> <div>If you control the client code, could you add some sleep before startt= ls and check if that fixes the issue?</div> </div> <div class=3D"gmail_extra"><br> <div class=3D"gmail_quote">On Wed, Dec 23, 2015 at 8:38 PM, Ulf Wiger <span= dir=3D"ltr"> <<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>>= ;</span> wrote:<br> <blockquote class=3D"gmail_quote" style=3D"PADDING-LEFT:1ex;MARGIN:0px 0px = 0px 0.8ex;BORDER-LEFT:#ccc 1px solid"> <div dir=3D"ltr"><span style=3D"FONT-SIZE:12px;FONT-FAMILY:Helvetica;COLOR:= rgb(0,0,0)">Hmm=E2=80=A6 I send this to erlang-bugs, but it didn=E2=80=99t = seem to get through.</span> <div style=3D"FONT-SIZE:12px;FONT-FAMILY:Helvetica;COLOR:rgb(0,0,0)"><br> </div> <div style=3D"FONT-SIZE:12px;FONT-FAMILY:Helvetica;COLOR:rgb(0,0,0)">When c= onnecting some Android software to an Erlang node using TLS, we sometimes (= about 1 in 3 or 4 times) get the following errors:<br> <br> 2015-12-22 15:31:00.772 [error] <0.210.0> gen_fsm <0.210.0> in = state hello terminated with reason: no function clause matching ssl_handsha= ke:update_handshake_history(undefined,=C2=A0<<1,0,0,175,3,1,86,121,22= 1,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247,96,14,32,123,176,...&g= t;>) line 450<br> <br> 15:31:00.783<dlink_tls_conn/327>dlink_tls_conn:terminate(): Reason: {= {function_clause,[{ssl_handshake,update_handshake_history,[undefined,<&l= t;1,0,0,175,3,1,86,121,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,2= 47,96,14,32,123,176,109,210,170,150,204,23,32,228,0,0,70,0,4,0,5,0,47,0,53,= 192,2,192,4,192,5,192,12,192,14,192,15,192,7,192,9,192,10,192,17,192,19,192= ,20,0,51,0,57,0,50,0,56,0,10,192,3,192,13,192,8,192,18,0,22,0,19,0,9,0,21,0= ,18,0,3,0,8,0,20,0,17,0,255,1,0,0,64,0,11,0,4,3,0,1,2,0,10,0,52,0,50,0,14,0= ,13,0,25,0,11,0,12,0,24,0,9,0,10,0,22,0,23,0,8,0,6,0,7,0,20,0,21,0,4,0,5,0,= 18,0,19,0,1,0,2,0,3,0,15,0,16,0,17>>],[{file,"ssl_handshake.erl&= quot;},{line,450}]},{tls_connection,'-next_state/4-fun-0-',3,[{file= ,"tls_connection.erl"},{line,458}]},{tls_connection,next_state,4,= [{file,"tls_connection.erl"},{line,467}]},{gen_fsm,handle_msg,7,[= {file,"gen_fsm.erl"},{line,518}]},{proc_lib,init_p_do_apply,3,[{f= ile,"proc_lib.erl"},{line,240}]}]},{gen_fsm,sync_send_all_state_e= vent,[<0.210.0>,{start,infinity},infinity]}}=C2=A0<br> <br> 2015-12-22 15:31:00.784 [error] <0.210.0> CRASH REPORT Process <0.= 210.0> with 0 neighbours exited with reason: no function clause matching= ssl_handshake:update_handshake_history(undefined,=C2=A0<<1,0,0,175,3= ,1,86,121,221,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247,96,14,32,1= 23,176,...>>) line 450 in gen_fsm:terminate/7 line 626<br> <br> 2015-12-22 15:31:00.785 [error] <0.209.0> gen_server <0.209.0> = terminated with reason: {{function_clause,[{ssl_handshake,update_handshake_= history,[undefined,<<1,0,0,175,3,1,86,121,221,42,209,19,198,53,3,42,9= 2,9,16,158,197,5,169,29,247,96,14,32,123,176,109,210,170,150,204,23,32,228,= 0,0,70,0,4,0,5,0,47,0,53,192,2,192,4,192,5,192,12,192,14,192,15,192,7,192,9= ,192,10,192,17,192,19,192,20,0,51,0,57,0,50,0,56,0,10,192,3,192,13,192,8,19= 2,18,0,22,0,19,0,9,0,21,0,18,0,3,0,8,0,20,0,17,0,255,1,0,0,64,0,11,0,4,3,0,= 1,2,0,10,0,52,0,50,0,14,0,13,0,25,0,11,0,12,0,24,0,9,0,10,0,22,0,23,0,8,0,.= ..>>],...},...]},...} in gen_fsm:sync_send_all_state_event/3 line 257<br> <br> 2015-12-22 15:31:00.786 [error] <0.209.0> CRASH REPORT Process <0.= 209.0> with 0 neighbours exited with reason: {{function_clause,[{ssl_han= dshake,update_handshake_history,[undefined,<<1,0,0,175,3,1,86,121,221= ,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247,96,14,32,123,176,109,21= 0,170,150,204,23,32,228,0,0,70,0,4,0,5,0,47,0,53,192,2,192,4,192,5,192,12,1= 92,14,192,15,192,7,192,9,192,10,192,17,192,19,192,20,0,51,0,57,0,50,0,56,0,= 10,192,3,192,13,192,8,192,18,0,22,0,19,0,9,0,21,0,18,0,3,0,8,0,20,0,17,0,25= 5,1,0,0,64,0,11,0,4,3,0,1,2,0,10,0,52,0,50,0,14,0,13,0,25,0,11,0,12,0,24,0,= 9,0,10,0,22,0,23,0,8,0,...>>],...},...]},...} in gen_server:terminate/7 line 826<br> <br> 2015-12-22 15:31:00.787 [error] <0.109.0> Supervisor tls_connection_s= up had child undefined started with {tls_connection,start_link,undefined} a= t <0.210.0> exit with reason no function clause matching=C2=A0ssl_han= dshake:update_handshake_history(undefined, <<1,0,0,175,3,1,86,121,221= ,42,209,19,198,53,3,42,92,9,16,158,197,5,169,29,247,96,14,32,123,176,...>= ;>) line 450 in context child_terminated<br> <br> <br> We run OTP Erlang/OTP 18 [erts-7.2] with ssl-7.2, and the erlang side has t= he following options:<br> <br> [{verify,verify_peer},<br> {certfile,"/home/.../device_cert.crt=E2=80=9D},<br> {keyfile,"/home/.../device_key.pem=E2=80=9D},<br> {cacertfile,"/home/.../root_cert.crt=E2=80=9D},<br> {verify_fun,{#Fun<dlink_tls_conn.65.24728257>,{'RSAPublicKey=E2= =80=99,...}}},<br> {partial_chain,#Fun<dlink_tls_conn.64.24728257>}]<br> <br> Basically, the verify_fun validates a self-signed cert<br> <a href=3D"https://github.com/PDXostc/rvi_core/blob/develop/components/dlin= k_tls/src/dlink_tls_conn.erl#L393" target=3D"_blank">https://github.com/PDX= ostc/rvi_core/blob/develop/components/dlink_tls/src/dlink_tls_conn.erl#L393= </a><br> <br> and the partial_chain fun most likely does much less than it should<br> <a href=3D"https://github.com/PDXostc/rvi_core/blob/develop/components/dlin= k_tls/src/dlink_tls_conn.erl#L421" target=3D"_blank">https://github.com/PDX= ostc/rvi_core/blob/develop/components/dlink_tls/src/dlink_tls_conn.erl#L421= </a><br> <br> On the Android side, we=E2=80=99re using Android 4.4.2 (API 19).<br> <br> It feels like a timing-related problem on the erlang side.<br> <br> Let me know if you need more information.<br> <br> BR,<br> Ulf W</div> </div> <br> _______________________________________________<br> erlang-bugs mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">erlang-bugs@erl= ang.org</a><br> <a href=3D"http://erlang.org/mailman/listinfo/erlang-bugs" rel=3D"noreferre= r" target=3D"_blank">http://erlang.org/mailman/listinfo/erlang-bugs</a><br> <br> </blockquote> </div> <br> <br clear=3D"all"> <div><br> </div> -- <br> <div> <div dir=3D"ltr"> <div><font face=3D"'courier new', monospace">Danil Zagoskin | <a hr= ef=3D"mailto:[email protected]" target=3D"_blank"> [email protected]</a></font></div> </div> </div> </div> </div> </div></div></div> </div> </div> </blockquote></div><br></div> --001a113f8ec21cb6a90527bef613-- --===============6789705141003137531== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ erlang-bugs mailing list [email protected] http://erlang.org/mailman/listinfo/erlang-bugs --===============6789705141003137531==--