Re: [EXTERN] Re: Re: Re: Definition of tokens in GP language

Hi, I'm there again :)

Thanks for sharing information. It motivates me more to write a lexer for Pygments.
Now I'm reading both files: the file lang.l in gp2c-repository and the function pari_lex() in the file anal.c.
If I understand correctly, I have two choices:

1) The function pari_lex() works fully correctly and can handle all conner cases in GP language,
but it's written by hand.
=> Porting to Python is not as easy as I wish (well writing a lexer was never easy :)).

2) The generated lexer from lang.l can now also handle all conner cases, but is not yet solid-rock as the function pari_lex() for now.
=> Porting in python, for example by using PLY[1] or RegexLexer[2] with State-Management, may be easier, but the Python lexer may not handle all corner cases?

Do I understand correctly? If so I think I take the 2nd choice, because my lexer is just for syntax highlighting. What do you think about?
I'll let you know when I were done with the lexer.

Best Wish!
HP Bui


[1] https://www.dabeaz.com/ply/ply.html#ply_nn21
[2] https://pygments.org/docs/lexerdevelopment/#changing-states


On 25/10/2025 14:25, Bill Allombert wrote:
> On Sat, Oct 25, 2025 at 12:46:41PM +0200, Bill Allombert wrote:
>> On Sat, Oct 25, 2025 at 11:37:29AM +0200, Hong-Phuc Bui wrote:
>>> Before I try to do something impossible: did someone try to rewrite the lexer in lex?
>>
>> Ah ah! Good idea. In fact, I did it already.
>> See gp2c, file src/lang.l
>>
>> https://pari.math.u-bordeaux.fr/cgi-bin/cgitweb.cgi?p=gp2c.git;a=blob_plain;f=src/lang.l;hb=HEAD
>>
>> but it fails in some corner cases, I need to fix it:
>>
>> echo "3!==6"| gp2c
>> stdin:1: syntax error, unexpected '='
>> Errors found: aborting...
> 
> I hopefully fixed it in gp2c GIT tree. The above link should point to the fixed version now.
> 
> Cheers,
> Bill

-- 
Hochschule für Technik und Wirtschaft des Saarlandes
University of Applied Sciences

Fakultät für Ingenieurwissenschaften
School of Engineering

Hong-Phuc Bui, M.Sc.
Informatik

Campus Alt-Saarbrücken
Goebenstraße 40
66117 Saarbrücken

+49 (0) 681 58 67 - 804
[email protected]
www.htwsaar.de