Re: [GSoC] Proposal: Modular LilyPond Parser (C++)

 >>It appears to me that you are proposing to replace an intentionally=20
flexible (and therefore complex) lexer with a simplified lexer that will=
=20
not have the same flexibility.

Thank you  for helping to ease some of the concerns. I can confirm that=20
my goal is neither to replace LilyPond's internal parser nor, even less=20
so, to restrict its current flexibility. My primary focus is on the=20
editor side. By coordinating our efforts, I hope to avoid unnecessary=20
work on both sides and prevent isolated solutions from emerging.

That said, I do want to take Carl's concerns seriously. The better my=20
lexer and parser become at working incrementally, the more likely it is=20
that someone might eventually get the idea of experimentally connecting=20
them to LilyPond. That idea would not necessarily have to come from me.=20
I would be lying=E2=80=94and naive=E2=80=94if I claimed that I have never =
considered the=20
possibility myself. At the very least, I would like to design the=20
architecture in such a way that doing so would not be completely=20
impossible. I'm not sure whether Erik Sandberg's argument was heading in=
=20
that direction; I didn't read it thoroughly.

My own problem arose quite naturally. I wanted to write an editor, and=20
at some point I found myself thinking: "Damn, if I keep going like this,=
=20
I'm going to end up reimplementing LilyPond=E2=80=94and that's exactly wha=
t I=20
never wanted to do... but I can't really avoid it."


 >>Furthermore, there are a range of different possible arguments to the=
=20
\time command; your preliminary specification is incomplete, although I=20
don't know enough about the proposed specification to know if that's=20
planned for being fixed.


To be honest, I still love my old Windows 7 machine running on 32-bit=20
hardware, which is why I'm still using LilyPond 2.18. The fact that the=20
following construct is now possible impressed me greatly:

\time #'(((1 2 3) . 8)((2 3/2) . 1/2))

Honestly, I promise you that I can highlight that faster than I can play=
=20
it on the piano (and I'm actually a fairly good pianist). ;o)

 >>But, you see, \time is not limited to a Voice context.  By default,=20
the time_signature_engraver is in the Staff context, not the Voice=20
context.  But it can be moved within the code.  So your "smart lexer" is=
=20
on its input specification inconsistent with actual LilyPond behavior.

To provide a deeper insight into my architecture regarding Carl's=20
example: I am well aware that writing `\score { \time ... }` or `\new=20
Voice { \time ... }` implicitly instantiates a Staff context behind the=20
scenes, and that it is ultimately the Staff engraver that processes the=20
time signature. However, since the time signature is intrinsically tied=20
to the temporal progression of music, I colloquially assigned it to the=20
Voice in my simplified table.

Even though everything can be expressed explicitly at the Staff level,=20
my lexer is extremely flexible in this regard=E2=80=94it simply comes down=
 to=20
how we choose to write the specification file.  The implicit creation of=
=20
contexts does add a layer of complexity when determining which contexts=20
accept a given literal, but it is not a fundamental roadblock. The logic o=
f

Context =E2=86=92 Engraver =E2=86=92 Consumed Types =E2=86=92 Literal

simply needs to be extended to account for these implicit context=20
insertions. Since the specification is completely decoupled from the=20
parser engine, adjusting these rules is trivial.

One more thing: in Qt I even built a window where the specification can=20
be edited live, and the syntax highlighting updates immediately as you=20
type. It doesn't get much more flexible than that.

Best regards
Christian