Re: [EXTERN] Re: Re: Definition of tokens in GP language

Before I try to do something impossible: did someone try to rewrite the lexer in lex?


On 24/10/2025 23:32, Bill Allombert wrote:
> On Fri, Oct 24, 2025 at 09:59:38PM +0200, Hong-Phuc Bui wrote:
>> Hi Bill,
>>
>> I took a look at the function. It is not really long, but I'm not sure about complexity:
>> Is it possible to recognize all tokens of GP by using regular express?
> 
> Strictly speaking it is not possible using regex due to ambiguities.
> For example
> 3!=6  is  3 != 6   (3 tokens)
> while
> 3!==6 is  3 ! == 6 (4 tokens)
> and
> 3!===6 is  3 ! === 6 (4 tokens)
> 
> There are other corner cases, depending how you define a token.
> 
> Note that all spaces are removed before parsing so for example
> a ! = b  is a != b (3 tokens)
> 
> Cheers,
> Bill.

-- 
Hochschule für Technik und Wirtschaft des Saarlandes
University of Applied Sciences

Fakultät für Ingenieurwissenschaften
School of Engineering

Hong-Phuc Bui, M.Sc.
Informatik

Campus Alt-Saarbrücken
Goebenstraße 40
66117 Saarbrücken

+49 (0) 681 58 67 - 804
[email protected]
www.htwsaar.de