Comparing inputs with source strings
[email protected] (Daniel Dehennin) Mon, 09 May 2016 16:53:15 +0200
| Newsgroups | perl.unicode |
|---|---|
| Message-ID | <[email protected]> |
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hello,
I tried to make my Perl5 code unicode compliant after reading a post on
stackoverflow[1].
As suggested in the post:
=E2=80=9Calways run incoming stuff through NFD and outbound stuff from =
NFC.=E2=80=9D
I got a hard time finding why my Test::More was failing but displaying
exactly the same strings for =E2=80=9Cgot=E2=80=9D and =E2=80=9Cexpected=E2=
=80=9D.
I finally check how UTF-8 sources are handled and found that they are in
NFC form, I run the following script:
#+begin_src perl
#!/usr/bin/env perl
use utf8;
use warnings;
use Test::More;
use Unicode::Normalize;
my $unistring =3D 'C=E2=80=99est une cha=C3=AEne unicode';
my @forms =3D ("NFD", "NFC", "NFKD", "NFKC");
for my $form (@forms) {
if ($unistring eq &$form($unistring)) {
print "UTF-8 source is in form '$form'\n";
}
}
#+end_src
and got:
#+begin_src
UTF-8 source is in form 'NFC'
UTF-8 source is in form 'NFKC'
#+end_src
So, the Test::More::is_deeply was trying to compare an input in NFD with
the expected string in NFC.
My code can use Unicode::Collate, but for all the code I did not write I
wonder if there is a way to handle it cleanly.
Or maybe I'm doing something wrong?
Regards.
Footnotes:=20
[1] https://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid=
-utf-8-by-default
=2D-=20
Daniel Dehennin
R=C3=A9cup=C3=A9rer ma clef GPG: gpg --recv-keys 0xCC1E9E5B7A6FE2DF
Fingerprint: 3E69 014E 5C23 50E8 9ED6 2AAD CC1E 9E5B 7A6F E2DF
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iL4EAREKAGYFAlcwpFtfFIAAAAAALgAoaXNzdWVyLWZwckBub3RhdGlvbnMub3Bl
bnBncC5maWZ0aGhvcnNlbWFuLm5ldDA5QUE4OTZBMEVDNkZEMDk5Mjc2ODdFODE2
QjJEMTMxQ0NBMDFBNEIACgkQFrLRMcygGksnYwD+PDYPwI8av4oe1rHqVPPG0JsJ
qEkeLXvCd1/ffggEx0sA/Rjvm717auBO4A39/StbhpnOvYKCBZSMhgAboW6IkBs0
=6U8/
-----END PGP SIGNATURE-----
--=-=-=--