Re: Python 3 and pygments-parser

--00000000000092e5c506181c8a29
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Shigio

On Thu, May 9, 2024 at 3:54=E2=80=AFAM Shigio YAMAGUCHI <[email protected]> wr=
ote:

> > What about converting the wrapper to consistently use bytearrays
> > instead of strings?
>
> I agree. I think that bytearray does not break compatibility
> with python2. Is it right?
>
...or alternatively, consistently use strings assuming all I/O was latin1
encoded. This is the case in other places in the wrapper and is unlikely to
put us in any worse situation than we already have. I believe this way it
would be possible to unify Python2 and Python3 code paths, too.


> > The original problem that made me look into
> > this was that =E2=80=98load_ctags_path()=E2=80=99 currently returns a b=
ytearray
> > (non-Win32) and the variable =E2=80=98UNIVERSAL_CTAGS=E2=80=99 is later=
 compared
> > with a string ('' !=3D b''). Under Python 2 this didn't matter.
>
> Do you mean that there is a bug in pygments_parser.py?
> If so, could you please explain the specific steps to reproduce it?
>

Compile global on a platform that is non-win32 (e.g. 'linux') and choose
Python3 as Python executable, a reasonable choice since Pygments itself,
which is imported, depends on Python3. Choose a pygments-parser
configuration and run gtags. Notice that GTAGS will contain no entries.

- In handle_requests(), the variable path is set to the result from
load_ctags_path(), which is a bytearray on the platform above.
- The following condition compares the bytearray path with the (empty)
string, which is always false.
- Therefore the global variable UNIVERSAL_CTAGS will always be overwritten
and end up as a bytearray, containing the configuration value of gtagscom.
- UNIVERSAL_CTAGS (bytearray) is compared with an empty string, which is
always false.
- The else-branch of the conditional is always executed, skipping
CtagsParser().

Best regards,
Marcus

--00000000000092e5c506181c8a29
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon=
t-size:small"></div></div><div style=3D"font-size:small" class=3D"gmail_def=
ault">Hi Shigio<br></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cl=
ass=3D"gmail_attr">On Thu, May 9, 2024 at 3:54=E2=80=AFAM Shigio YAMAGUCHI =
&lt;<a href=3D"mailto:[email protected]">[email protected]</a>&gt; wrote:<br></di=
v><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;borde=
r-left:1px solid rgb(204,204,204);padding-left:1ex">
&gt; What about converting the wrapper to consistently use bytearrays<br>
&gt; instead of strings?<br>
<br>
I agree. I think that bytearray does not break compatibility<br>
with python2. Is it right?<br></blockquote><div><div style=3D"font-size:sma=
ll" class=3D"gmail_default"></div><div style=3D"font-size:small" class=3D"g=
mail_default">...or alternatively, consistently use strings assuming all I/=
O was latin1 encoded. This is the case=20
in other places in the wrapper and is unlikely to put us in any worse=20
situation than we already have. I believe this way it would be possible=20
to unify Python2 and Python3 code paths, too.</div></div><div>=C2=A0<br></d=
iv><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord=
er-left:1px solid rgb(204,204,204);padding-left:1ex">
&gt; The original problem that made me look into<br>
&gt; this was that =E2=80=98load_ctags_path()=E2=80=99 currently returns a =
bytearray<br>
&gt; (non-Win32) and the variable =E2=80=98UNIVERSAL_CTAGS=E2=80=99 is late=
r compared<br>
&gt; with a string (&#39;&#39; !=3D b&#39;&#39;). Under Python 2 this didn&=
#39;t matter.<br>
<br>
Do you mean that there is a bug in pygments_parser.py?<br>
If so, could you please explain the specific steps to reproduce it?<br></bl=
ockquote><div><br></div><div style=3D"font-size:small" class=3D"gmail_defau=
lt">Compile global on a platform that is non-win32 (e.g. &#39;linux&#39;) a=
nd choose Python3 as Python executable, a reasonable choice since Pygments =
itself, which is imported, depends on Python3. Choose a pygments-parser con=
figuration and run gtags. Notice that GTAGS will contain no entries.</div><=
div style=3D"font-size:small" class=3D"gmail_default"><br></div><div style=
=3D"font-size:small" class=3D"gmail_default">- In handle_requests(), the va=
riable path is set to the result from load_ctags_path(), which is a bytearr=
ay on the platform above.</div><div style=3D"font-size:small" class=3D"gmai=
l_default">- The following condition compares the bytearray path with the (=
empty) string, which is always false.</div><div style=3D"font-size:small" c=
lass=3D"gmail_default">- Therefore the global variable UNIVERSAL_CTAGS will=
 always be overwritten and end up as a bytearray, containing the configurat=
ion value of gtagscom.</div><div style=3D"font-size:small" class=3D"gmail_d=
efault">- UNIVERSAL_CTAGS (bytearray) is compared with an empty string, whi=
ch is always false.</div><div style=3D"font-size:small" class=3D"gmail_defa=
ult">- The else-branch of the conditional is always executed, skipping Ctag=
sParser().</div><div style=3D"font-size:small" class=3D"gmail_default"><br>=
</div><div style=3D"font-size:small" class=3D"gmail_default">Best regards,<=
/div><div style=3D"font-size:small" class=3D"gmail_default">Marcus</div><di=
v style=3D"font-size:small" class=3D"gmail_default"><br></div></div></div>

--00000000000092e5c506181c8a29--