Re: Backtracking through the source

At 09:58 AM 11/29/00 -0800, Steve Fink wrote:
>As soon as the lexer sees "s#", it
>starts treating # as a delimiter -- it doesn't need to conditionally
>treat the # as either a delimiter or a comment. (Especially since
>there's nothing following it that could resolve the ambiguity!)

One thing I've been thinking about is setting up a host of symbolic 
representations of things like comment characters and end-of-statement 
delimiters and things. That way, rather than hardcoding # as the comment 
character (which it sometimes isn't) we look for the generic "comment 
sequence". Makes things like q## more generically handleable, and it also 
means that if we put in, say, c_mode.pm, it can set the close-comment 
sequence to */.

It seems we'll need to do is weight various lexical constructs, so that the 
open-quote sequence is checked for before the comment sequence. (Unless we 
want to do some sort of probabalistic thing, or turn the whole parser into 
a big neural net or something...)

					Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
[email protected]                         have teddy bears and even
                                      teddy bears get drunk