Re: UTF-8 encoding & decoding

On 05/05/2016 08:37 AM, Pali Roh=C3=A1r wrote:
> Hi!
>
> I though that I understand UTF-8 encoding/decoding done in perl until I
> looked into source code of Encode package... (exactly sub encode_utf8)
>
> Before... I only read description of Encode package (not source code):
> https://metacpan.org/pod/Encode#UTF-8-vs.-utf8-vs.-UTF8
>
> I tried to find some more information (ideally those which answer my
> question) but without success. Can you help me? My questions are:
>
> 1. What is difference between those two calls?
>
>   utf8::encode($str);
>
> and
>
>   $str =3D Encode::encode('utf8', $str);
>
> 2. What is difference between those?
>
>   utf8::decode($str);
>   $str =3D Encode::decode_utf8($str);

Each pair of functions is supposed to do essentially the same thing. I=20
have not studied them to know what subtle differences there may be.
>
> 3. Where is implementation of utf8::encode/decode functions? It is not
> in utf8.pm, nor in utf8_heavy.pl and also not in unicore/Heavy.pl. And
> what those functions doing?

The implementation is in universal.c.  But these are just wrappers for=20
sv_utf8_encode and sv_utf8_decode, which are implemented in sv.c.  Their=20
documentation is in perlapi.  It should match the documentation of=20
utf8::decode and utf8::encode, whose documentation is in utf8.pm.  (I=20
myself have a hard time mapping the names chosen for these operations=20
with what they actually do)
>