Re: Python 3 and pygments-parser

--000000000000debdde0617ed6f99
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Shigio

Never mind, the idea with =E2=80=98locale.getpreferredencoding(False)=E2=80=
=99 isn't going
to work anyway, for the lack of multibyte support and differences between
user locale and e.g. filename encoding, etc.
What about converting the wrapper to consistently use bytearrays instead of
strings? The original problem that made me look into this was that =E2=80=
=98
load_ctags_path()=E2=80=99 currently returns a bytearray (non-Win32) and th=
e
variable =E2=80=98UNIVERSAL_CTAGS=E2=80=99 is later compared with a string =
('' !=3D b'').
Under Python 2 this didn't matter.

Best regards,
Marcus

On Wed, May 8, 2024 at 2:17=E2=80=AFAM Shigio YAMAGUCHI <[email protected]> wr=
ote:

> Hi Marcus,
>
> > The method =E2=80=98load_ctags_path()=E2=80=99 doesn't work with Python=
 3
> > (unless perhaps with Win32/latin1).
> > In other places the encoding has been fixed to =E2=80=98latin1=E2=80=99=
 as well.
>
> It doesn't matter that the encoding is latin1, because Global
> doesn't support multi-byte code set.
>
>         [FAQ]
>         Q10. Does Global support multi-byte code set?
>              Which character code set is supported?
>
>         A10. Global doesn't support multi-byte character code set yet.
>              Global supports only ASCII and ASCII super-sets.
>
> > At least for =E2=80=98load_gtags_path()=E2=80=99 I'd probably change th=
e encoding
> > to the current locale with =E2=80=98locale.getpreferredencoding(False)=
=E2=80=99.
>
> What does it accomplish?
>
> > Python 2 support has been dropped from Pygments more than
> > four years ago. Does it make sense to maintain Python 2
> > compatibility in the pygments-parser?
>
> Though maintaining compatibility is important, if we can achieve
> something that users will be happy with, it is not the top priority.
> I'd appreciate it if you could let me know what you're trying
> to achieve.
>
> Regards,
> Shigio
> --
> Shigio YAMAGUCHI <[email protected]>
> PGP fingerprint:
> 26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB
>

--000000000000debdde0617ed6f99
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:small">Hi =
Shigio</div><div class=3D"gmail_default" style=3D"font-size:small"><br></di=
v><div class=3D"gmail_default" style=3D"font-size:small">Never mind, the id=
ea with =E2=80=98locale.getpreferredencoding(False)=E2=80=99 isn&#39;t goin=
g to work anyway, for the lack of multibyte support and differences between=
 user locale and e.g. filename encoding, etc.<br></div><div class=3D"gmail_=
default" style=3D"font-size:small">What about converting the wrapper to con=
sistently use bytearrays instead of strings? The original problem that made=
 me look into this was that =E2=80=98<span class=3D"gmail-im">load_ctags_pa=
th()</span>=E2=80=99 currently returns a bytearray (non-Win32) and the vari=
able =E2=80=98UNIVERSAL_CTAGS=E2=80=99 is later compared with a string (&#3=
9;&#39; !=3D b&#39;&#39;). Under Python 2 this didn&#39;t matter.</div><div=
 class=3D"gmail_default" style=3D"font-size:small"><br></div><div class=3D"=
gmail_default" style=3D"font-size:small">Best regards,</div><div class=3D"g=
mail_default" style=3D"font-size:small">Marcus<br></div></div><br><div clas=
s=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, May 8, 2024=
 at 2:17=E2=80=AFAM Shigio YAMAGUCHI &lt;<a href=3D"mailto:[email protected]">=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" st=
yle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padd=
ing-left:1ex">Hi Marcus,<br>
<br>
&gt; The method =E2=80=98load_ctags_path()=E2=80=99 doesn&#39;t work with P=
ython 3<br>
&gt; (unless perhaps with Win32/latin1).<br>
&gt; In other places the encoding has been fixed to =E2=80=98latin1=E2=80=
=99 as well.<br>
<br>
It doesn&#39;t matter that the encoding is latin1, because Global<br>
doesn&#39;t support multi-byte code set.<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 [FAQ]<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Q10. Does Global support multi-byte code set?<b=
r>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Which character code set is=
 supported?<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 A10. Global doesn&#39;t support multi-byte char=
acter code set yet.<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Global supports only ASCII =
and ASCII super-sets.<br>
<br>
&gt; At least for =E2=80=98load_gtags_path()=E2=80=99 I&#39;d probably chan=
ge the encoding<br>
&gt; to the current locale with =E2=80=98locale.getpreferredencoding(False)=
=E2=80=99.<br>
<br>
What does it accomplish?<br>
<br>
&gt; Python 2 support has been dropped from Pygments more than<br>
&gt; four years ago. Does it make sense to maintain Python 2<br>
&gt; compatibility in the pygments-parser?<br>
<br>
Though maintaining compatibility is important, if we can achieve<br>
something that users will be happy with, it is not the top priority.<br>
I&#39;d appreciate it if you could let me know what you&#39;re trying<br>
to achieve.<br>
<br>
Regards,<br>
Shigio<br>
-- <br>
Shigio YAMAGUCHI &lt;<a href=3D"mailto:[email protected]" target=3D"_blank">sh=
[email protected]</a>&gt;<br>
PGP fingerprint:<br>
26F6 31B4 3D62 4A92 7E6F=C2=A0 1C33 969C 3BE3 89DD A6EB<br>
</blockquote></div>

--000000000000debdde0617ed6f99--