Re: Wow, Lisp Reader !!

Yes, a true Lexical Analyzer is how I would prefer to approach things. I did that for my NML language written in SML and OCaml. 

I remember being taken aback many decades ago, when I saw the Lisp approach of vectoring on characters, one by one. But Pascal B. just showed how that can be useful by attacking the digit chars as parse start points. I had never ventured with Macro Characters, but only Dispatch Macro Characters.

And the key, as he shows, is to fall back to READ-CHAR and UNREAD-CHAR inside the Macro Character handlers. Do not try to rely on READ or READ-FROM-STRING.

> On May 24, 2026, at 05:05, Paul Tarvydas <[email protected]> wrote:
> 
> FWIW, my knee-jerk reaction is to think that I would want to use ESRAP (https://scymtym.github.io/esrap/) in conjunction with the read table. I haven't tried it, since I just use PEG parsers and avoid the issue(s) completely.
> 
> I see this as a lexer vs. parser issue. You can't (or, at least, shouldn't) use just a lexer (like regex, say) to build every aspect of a parser.
> 
> Thing vs. 'Thing vs. Thing' should be lexed into something like (respectively)
> 
> [{chars: "Thing"}] 
> [{delimiter: singlequote}, {chars: "Thing"}]
> [{chars: "Thing"}, {delimiter: singlequote}]. 
> Then, a parser should take over with rules like:
> 
> sym = <chars> <delimiter>?
> quotedsym = <delimiter: singlequote> <sym>
> If all you've got is a lexer, or just a readtable, then you have to begin using politically incorrect phrases, like "state change". This is what Forth does. Forth lets you ingest one "word" then change the state of the world in an irreversible manner, i.e. you switch to another lexer. The new lexer must know when to quit and switch back to the original lexer, say by pushing and popping a stack (like a parser does). [opinion: we could do with a lot more non-functional thinking. I took a look at Forth Haiku and observe that what takes 2 lines of Forth gets translated into about 100 lines of GLSL]
> 
> 
> 
> pt
> 
> 
>