Re: Retrieving components of a regexp with ocamllex

« What lexers do is to split the input text in atomic units called "lexemes ». A rule should parse one lexeme, not more. »

Yes, but there is no uniform rule to decide what is a lexeme and what is not. You wrote for example that `xyz` is a lexeme, but why not consider it a compound of the three lexemes `, xyz and `? In the OCaml syntax, for example, `xyz` is not a lexeme. It all depends on context and particularities of the language one is writing a compiler for. 

In several cases, one has the choice between performing the treatment in the lexer part or the parser part.
What initially motivated me to look into this was that  my parser part was growing fast in complexity and size while my lexer part stayed simple and short, so it seemed more balanced to put more stuff into the lexer part  when possible.

« You should read up on how to structure a lexer+parser combination, or look at existing examples. »

I’ve already done all that of course, so could you please be more specific.