Re: StringBuilder Extension: IsQuotedBy

Peter Obiefuna <[email protected]> wrote:

> >> Daniel said: developing complex parsers without regexp isn't for the
> >> faint-hearted =)
> 
> Do people still do that? And why would anyone want to do that? If you ask me
> it will amount to whipping your own home-grown char-snake-and-ladder
> "alphabet state machine" AKA a regex engine.

Only if it needs to be completely general. Well-defined languages don't
need the flexibility of a full regex engine. A hand-written DFA that
matches tokens is easy to write and pretty easy to modify; the current
execution point represents the current state of the DFA, and simple
switches, if statements and loops form the basic alternations and Kleene
closures of your state transitions.

> Wouldn't every lexer have to do this to identify tokens? It seems like an
> obligatory step for most really big parsing workflows.

Any single given regex terminates in a single state, "matched". A
tokenizer has multiple ending states, one for each token. This is the
key difference between most third-party regex libraries and the
requirements of people writing parsers for well-defined languages.

-- Barry

-- 
http://barrkel.blogspot.com/

===================================
This list is hosted by DevelopMentor®  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com