Re: DBD::Pg && handling of UTF-8 char columns

--0000000000004aa2a20594ecb0e2
Content-Type: text/plain; charset="UTF-8"

On Mon, Oct 14, 2019 at 10:46 PM Matthias Apitz <[email protected]> wrote:

> i.e. from the database PG server is coming the code point correctly as
> (octal) \303\244 which is the same as \xc3\xa4. And Perl mangles this to
>
> [UTF8 "P\x{e4}dagogische Hochschule Weingarten"]
>
> which is IMHO not correct and causing all this confusion.
>
> We have to deal with this in our perl code. It's not a PostrgreSQL
> problem.
>

We use UTF8 extensively with Pg and Perl and have no issues,
so I suspect there's a configuration issue somewhere.

And yes, I don't think you should be worrying about how Perl encodes things
internally.

From perlunifaq:

> *I lost track; what encoding is the internal format really?*

It's good that you lost track, because you shouldn't depend on the internal
> format being any specific encoding. But since you asked: by default, the
> internal format is either ISO-8859-1 (latin-1), or utf8, depending on the
> history of the string. On EBCDIC platforms, this may be different even.
> Perl knows how it stored the string internally, and will use that
> knowledge when you encode . In other words: don't try to find out what the
> internal encoding for a certain string is, but instead just encode it into
> the encoding that you want.


https://perldoc.perl.org/perlunifaq.html#INTERNALS

Maurice

--0000000000004aa2a20594ecb0e2
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Mon, Oct 14, 2019 at 10:46 PM Matt=
hias Apitz &lt;<a href=3D"mailto:[email protected]">[email protected]</a>&gt;=
 wrote:</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px =
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
i.e. from the database PG server is coming the code point correctly as<br>
(octal) \303\244 which is the same as \xc3\xa4. And Perl mangles this to <b=
r>
<br>
[UTF8 &quot;P\x{e4}dagogische Hochschule Weingarten&quot;]<br>
<br>
which is IMHO not correct and causing all this confusion.<br>
<br>
We have to deal with this in our perl code. It&#39;s not a PostrgreSQL<br>
problem.<br></blockquote><div><br></div><div>We use UTF8 extensively with P=
g and Perl and have no issues,</div><div>so I suspect there&#39;s a configu=
ration issue somewhere.</div><div><br></div><div>And yes, I don&#39;t think=
 you should be worrying about how Perl encodes things internally.</div><div=
><br></div><div>From perlunifaq:</div><blockquote class=3D"gmail_quote" sty=
le=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddi=
ng-left:1ex"><b>I lost track; what encoding is the internal format really?<=
/b></blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">It&#39;s =
good that you lost track, because you shouldn&#39;t depend on the internal =
format being any specific encoding. But since you asked: by default, the in=
ternal format is either ISO-8859-1 (latin-1), or utf8, depending on the his=
tory of the string. On EBCDIC platforms, this may be different even.<br>Per=
l knows how it stored the string internally, and will use that knowledge wh=
en you encode . In other words: don&#39;t try to find out what the internal=
 encoding for a certain string is, but instead just encode it into the enco=
ding that you want.</blockquote><div><br></div><div><a href=3D"https://perl=
doc.perl.org/perlunifaq.html#INTERNALS">https://perldoc.perl.org/perlunifaq=
.html#INTERNALS</a>=C2=A0</div><div><br></div><div>Maurice</div></div></div=
>

--0000000000004aa2a20594ecb0e2--