(no subject)
Vlad Dumitrescu <[email protected]> Fri, 30 Jan 2015 09:52:15 +0100
| Newsgroups | gmane.comp.lang.erlang.patches |
|---|---|
| Message-ID | <CAA-EFXu-XcR=uhO7CObD6_z8kZwUy6GS8JL-LMcmWVMOtqBJ7g@mail.gmail.com> |
--===============1449797997646142942== Content-Type: multipart/alternative; boundary=001a11c2691edf454f050ddab926 --001a11c2691edf454f050ddab926 Content-Type: text/plain; charset=UTF-8 OTP-11660: make eunit unicode safe The most critical part is eunit_surefire, because the generated XML must be correct UTF-8, otherwise tools will refuse to work with it. The tricky part is that the input can contain mixed encodings and we can't know which encoding applies to each text chunk. This is because input can be from io:format in the tests. Also, different test modules might be encoded differently and that can't be detected at runtime. We have to avoid outputting both bad utf-8 and doubly encoded utf-8. The simplest solution is to let the encoder skip characters it chokes on, replacing them with '?'. > git fetch git://github.com/vladdu/otp.git eunit_unicode_OTP11660 > > https://github.com/vladdu/otp/compare/erlang > :master...eunit_unicode_OTP11660 > https://github.com/vladdu/otp/compare/erlang > :master...eunit_unicode_OTP11660.patch regards, Vlad --001a11c2691edf454f050ddab926 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div di= r=3D"ltr"><div style><div style><div dir=3D"ltr" style>OTP-11660: make euni= t unicode safe</div><div dir=3D"ltr" style><br></div><div dir=3D"ltr" style= >The most critical part is eunit_surefire, because the generated XML</div><= div dir=3D"ltr" style>must be correct UTF-8, otherwise tools will refuse to= work with it.</div><div dir=3D"ltr" style><br></div><div dir=3D"ltr" style= >The tricky part is that the input can contain mixed encodings and we</div>= <div dir=3D"ltr" style>can't know which encoding applies to each text c= hunk. This is because</div><div dir=3D"ltr" style>input can be from io:form= at in the tests. Also, different test modules</div><div dir=3D"ltr" style>m= ight be encoded differently and that can't be detected at runtime. We</= div><div dir=3D"ltr" style>have to avoid outputting both bad utf-8 and doub= ly encoded utf-8.</div><div dir=3D"ltr" style><br></div><div dir=3D"ltr" st= yle>The simplest solution is to let the encoder skip characters it chokes</= div><div dir=3D"ltr" style>on, replacing them with '?'.</div><div d= ir=3D"ltr" style=3D"font-family:arial,sans-serif;font-size:13px"><br></div>= </div><div style=3D"font-family:arial,sans-serif;font-size:13px">=C2=A0</di= v><blockquote class=3D"gmail_quote" style=3D"font-family:arial,sans-serif;f= ont-size:13px;margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-co= lor:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> git fetch git://<a href=3D"http://github.com/vladdu/otp.git" target=3D"_bla= nk">github.com/vladdu/otp.git</a>=C2=A0eunit_unicode_OTP11660<br></blockquo= te><div style=3D"font-family:arial,sans-serif;font-size:13px">=C2=A0</div><= blockquote class=3D"gmail_quote" style=3D"font-family:arial,sans-serif;font= -size:13px;margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color= :rgb(204,204,204);border-left-style:solid;padding-left:1ex"> <a href=3D"https://github.com/vladdu/otp/compare/">https://github.com/vladd= u/otp/compare/</a><span style=3D"color:rgb(34,34,34);background-color:rgb(2= 55,255,204)">erlang</span>:master...eunit_unicode_OTP11660<br> <a href=3D"https://github.com/vladdu/otp/compare/">https://github.com/vladd= u/otp/compare/</a><span style=3D"color:rgb(34,34,34);background-color:rgb(2= 55,255,204)">erlang</span>:master...eunit_unicode_OTP11660.patch</blockquot= e> <div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">regards,</div><div sty= le=3D"font-family:arial,sans-serif;font-size:13px">Vlad</div><div style=3D"= font-family:arial,sans-serif;font-size:13px"><br></div></div></div></div> <br></div> <br></div> <br></div> --001a11c2691edf454f050ddab926-- --===============1449797997646142942== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ erlang-patches mailing list [email protected] http://erlang.org/mailman/listinfo/erlang-patches --===============1449797997646142942==--