Re: need help with utf-8

--5eayn2dnxzok477j
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2024-12-18 09:41:13 -0500, Shaomei Liu wrote:
> if you happen to have an example to show the new behavior is more
> "correct",

I haven't been on any of the main Perl mailing-lists or newsgroups for a
long time, so this may be outdated, but the general idea is that the
dichotomy between byte strings and character strings was a mistake and
that two strings which compare equal should hehave the same whenever
possible. The difference is just too subtle and error-prone.

In particular, the string you created in your test script was a byte
string with three bytes ("\xe2\x80\x9C"). That string has length 3 and
it will compare equal to the string with the three characters=20
U+00E2 LATIN SMALL LETTER A WITH CIRCUMFLEX, U+0080 PADDING CHARACTER,
U+009C STRING TERMINATOR. So it stands to reason that it should be
treated the same as that 3 character string, and the varchar stored in
the database should also be 3 characters long and not just a single
character, just because it happens to be a byte sequence which happens
to match that character's UTF-8 encoding.


> On Wed, Dec 18, 2024 at 8:53=E2=80=AFAM Felipe Gasper <felipe@felipegaspe=
r.com> wrote:
> >
> > Do we know, in fact, why this changed?
> >
> > The new behaviour may be =E2=80=9Cmore correct=E2=80=9D, but it=E2=80=
=99ll still subtly
> > break a bunch of stuff that worked fine before.

True.

But it should probably also be noted that Redhat 7 was released in 2014 and
Redhat 8 in 2019. So the "new behaviour" is now between 5 and 10 years
old.

I'm too lazy to track down the release which introduced the change
(especially since there seems to be a huge gap in the history on CPAN),
but I would expect that to be mentioned in the release notes at the
time.

        hp


--=20
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | [email protected]         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

--5eayn2dnxzok477j
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmdkMR8ACgkQ8g5IURL+
KF2bsRAAiPStqFWCkAl1oI9ui2M1Ff5TRGblyfvFtZhsG9+kjub2VXWNW8VNBZZp
3Xl2q5rZ57QgBtSiiopAV29PT0yXkujg8wJSVlYZn7RJN+/cNlu/JWo5tDcm2Av6
z7XVYjIbnfDLhKo32vRhtgL8QXMSEOeTRVkQECqHQprZ/LKtVQU81+u5OBVNLA8Q
HAXV3CTdWl8rTwU5XwDYtDbM6oacG3JDPXEI8euAYfMZ9w7TT8RfONJEZy4vbwYJ
OyE8V/zFJGgpKY6ntdVvXjaTWcc7hM9UN1hWZBTMhCb+EVovg38TS4tLxhefN+0I
NIw93Rk7c/dCgan4sy6E+ll14x8oqMufZ5rNmOujaHlEtzZxBn3uzyJH6YsErh6v
AVdySLAmRZA6Zgx0/osYcKx+D9hUx+/CdGMlZ3K4xerYlJAUV8ySo6TGAyXINEok
GNsFE/bKvjEq7PJO3O3J96Vh8VPYzsfZ2DMyR3FuhnvwKrdU2KYCxA4hOs6KVzzG
3F/y5kAICC7jTa2ewFQAkTNWYVzVu7iQCq4m/FmLoIvtV5ShQFltvTuhCYOuqXiC
KEDNVLb6PWuhZP8iVZbbqiskyZMNOWEuFa7jDd1wOVI0V8mDIXoCdqpcqYMRtwgO
BTE5v0Go3SkoVjWhJqT5bXpD1UEAx8wwhq59EiJuX+hTW5qKXhA=
=iJP3
-----END PGP SIGNATURE-----

--5eayn2dnxzok477j--