Re: Python 3 and pygments-parser

Hi Marcus,

>> Do you mean that there is a bug in pygments_parser.py?
>> If so, could you please explain the specific steps to reproduce it?
>
> Compile global on a platform that is non-win32 (e.g. 'linux') and
> choose Python3 as Python executable, a reasonable choice since Pygments
> itself, which is imported, depends on Python3. Choose a pygments-parser
> configuration and run gtags. Notice that GTAGS will contain no entries.

It doesn't seem to reproduce in my environment. GTAGS and GRTAGS
have symbols. Is there a difference between the python versions?

$ sw_vers
ProductName: macOS
ProductVersion: 14.2.1
BuildVersion: 23C71
$ cat gtags.conf
default:\
:ctagscom=3D/opt/local/bin/uctags:\
:langmap=3DC\:.c.h:\
:gtags_parser=3DC\:/usr/local/lib/gtags/pygments-parser.la:
$ head -1 /usr/local/share/gtags/script/pygments_parser.py
#!/opt/local/bin/python3.12
$ /opt/local/bin/python3.12 --version
Python 3.12.1
$ /usr/local/bin/gtags --explain
 - File 'libdb/bt_get.c' is handled as follows:
        suffix:   |.c|
        language: |C|
        parser:   |parser|
        library:  |/usr/local/lib/gtags/pygments-parser.la|
(omitted)
$ /usr/local/bin/global -x get_dbpath
get_dbpath        367 libutil/getdbpath.c get_dbpath(void)
$ /usr/local/bin/global -x get_dbpath -r
get_dbpath        413 global/global.c  dbpath =3D get_dbpath();
get_dbpath        199 gozilla/gozilla.c dbpath =3D get_dbpath();
get_dbpath       1352 htags/htags.c    strlimcpy(dbpath, get_dbpath(),
sizeof(dbpath));
get_dbpath       1468 htags/htags.c    if (get_dbpath())
get_dbpath       1469 htags/htags.c    message(" Using %s/GTAGS.",
get_dbpath());
get_dbpath         30 libutil/getdbpath.h const char *get_dbpath(void);

Regards,
Shigio

On Sat, May 11, 2024 at 2:05=E2=80=AFAM Marcus Harnisch
<[email protected]> wrote:
>
> Hi Shigio
>
> On Thu, May 9, 2024 at 3:54=E2=80=AFAM Shigio YAMAGUCHI <[email protected]> =
wrote:
>>
>> > What about converting the wrapper to consistently use bytearrays
>> > instead of strings?
>>
>> I agree. I think that bytearray does not break compatibility
>> with python2. Is it right?
>
> ...or alternatively, consistently use strings assuming all I/O was latin1=
 encoded. This is the case in other places in the wrapper and is unlikely t=
o put us in any worse situation than we already have. I believe this way it=
 would be possible to unify Python2 and Python3 code paths, too.
>
>>
>> > The original problem that made me look into
>> > this was that =E2=80=98load_ctags_path()=E2=80=99 currently returns a =
bytearray
>> > (non-Win32) and the variable =E2=80=98UNIVERSAL_CTAGS=E2=80=99 is late=
r compared
>> > with a string ('' !=3D b''). Under Python 2 this didn't matter.
>>
>> Do you mean that there is a bug in pygments_parser.py?
>> If so, could you please explain the specific steps to reproduce it?
>
>
> Compile global on a platform that is non-win32 (e.g. 'linux') and choose =
Python3 as Python executable, a reasonable choice since Pygments itself, wh=
ich is imported, depends on Python3. Choose a pygments-parser configuration=
 and run gtags. Notice that GTAGS will contain no entries.
>
> - In handle_requests(), the variable path is set to the result from load_=
ctags_path(), which is a bytearray on the platform above.
> - The following condition compares the bytearray path with the (empty) st=
ring, which is always false.
> - Therefore the global variable UNIVERSAL_CTAGS will always be overwritte=
n and end up as a bytearray, containing the configuration value of gtagscom=
.
> - UNIVERSAL_CTAGS (bytearray) is compared with an empty string, which is =
always false.
> - The else-branch of the conditional is always executed, skipping CtagsPa=
rser().
>
> Best regards,
> Marcus
>


--=20
Shigio YAMAGUCHI <[email protected]>
PGP fingerprint:
26F6 31B4 3D62 4A92 7E6F  1C33 969C 3BE3 89DD A6EB