Re: Custom color lexer for DrRacket that requires lookahead

On Mon, May 30, 2016 at 2:32 PM, Alexis King <[email protected]> wrote:
>
>> On May 30, 2016, at 12:23, Alexis King <[email protected]> wrote:
>>
>> Now the colorer attempts to lex the slash alone as a new token without
>> attempting to re-lex “foo”. Since a forward slash is not valid at the
>> beginning of an identifier, it is colored as an error.
>>
>> Basically, when I type a forward slash, I want the preceding token to be
>> re-lexed. I don’t need anything more than that... just a single
>> character of lookahead is necessary, effectively.
>
> Actually, I realized just after sending this that my description is not
> the full picture. I think the confusion probably stems from that the
> colorer does, in fact attempt to re-lex “foo” once the forward slash is
> typed. However, since a trailing slash is illegal at the end of an
> identifier as well, it’s still lexed as an error. Now, I keep typing:
>
>   foo/bar
>          ^
>
> This presumably causes the slash to be re-lexed, but since it’s still
> illegal at the beginning of a token, the lexer still fails. My guess
> here was that I needed to include a backup-distance on the error token,
> but that doesn’t seem to help, so I’m unsure how to properly indicate
> the need to re-lex the entire “foo/bar” token once “bar” is added.

I'm wondering why the entire thing isn't a single token, actually. So,
when you type any of these:

  f
  fo
  foo
  foo/
  foo/b
  foo/ba
  foo/bar

your lexer would always say that they are a single token. The fourth
one would be an error token, but still a single token.

Robby

-- 
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
For more options, visit https://groups.google.com/d/optout.