Re: Retrieving components of a regexp with ocamllex

I merely meant that your current pair rule is not a good idea.

The reason why you don't read `xyz` as "`"; "xyz"; "`" is that the xyz part
has no lexical structure in itself, apart from "being inside quotes". At
least if you also allow `xy z`, etc.

If your parser is too complex, you should have a look at Menhir and its
parametrized rules as a way to manage complexity (as opposed to yacc; if
you are already using something else, that is fine).

On Sun, May 10, 2015 at 8:48 PM, [email protected] [ocaml_beginners]
<[email protected]> wrote:

>
>
>
> « What lexers do is to split the input text in atomic units called
> "lexemes ». A rule should parse one lexeme, not more. »
>
> Yes, but there is no uniform rule to decide what is a lexeme and what is
> not. You wrote for example that `xyz` is a lexeme, but why not consider it
> a compound of the three lexemes `, xyz and `? In the OCaml syntax, for
> example, `xyz` is not a lexeme. It all depends on context and
> particularities of the language one is writing a compiler for.
>
> In several cases, one has the choice between performing the treatment in
> the lexer part or the parser part.
> What initially motivated me to look into this was that  my parser part was
> growing fast in complexity and size while my lexer part stayed simple and
> short, so it seemed more balanced to put more stuff into the lexer part
> when possible.
>
> « You should read up on how to structure a lexer+parser combination, or
> look at existing examples. »
>
> I’ve already done all that of course, so could you please be more
> specific.
>
> 
>