SMIE: general feedback and a half-baked idea

I just sent a message about a bunch of specific problems I’ve been
having while trying to use SMIE to indent Roto code.  In this separate
thread, I'd like to zoom out a little and offer some feedback on the
overall experience of working with SMIE for this language.

First, let me say that if SMIE didn’t exist I probably wouldn’t have
attempted to write a major mode for Roto at all.  I’ve seen how
complicated the hand-written indentation engines in Emacs get, and this
is already multiple levels of yak-shaving away from the thing I
actually wanted to do; without the *promise* of being able to specify
indentation rules mostly declaratively, a mode wouldn’t have seemed
worth the effort.

That said, it has been a very frustrating experience so far.  I think
I can boil all the problems I’ve been having down to two things: a
documentation issue, and a limitation of OPGs that I have an idea for
how to solve.

The documentation issue, first, is that it’s really unclear *what to
leave out* of a SMIE BNF grammar.  I’m a compiler guy, I can read over
https://roto.docs.nlnetlabs.nl/en/stable/ and distill it down to an
approximate *complete* BNF for the language, but the impression I have
so far is you shouldn’t actually give SMIE a complete BNF, you should
only tell it about things that are directly relevant to indentation
and sexp movement.  The trouble is figuring out where to draw the
line.  What I could have used -- and would be willing to help write --
is a series of examples, starting from the complete BNF for *all of* a
simple but non-toy language (Roto itself might serve this purpose, in
fact), and walking the reader through what to remove, what to fudge,
and what you actually have to put clever hacks into the lexer to deal
with.

[Tangent: Speaking of putting clever hacks into the lexer, the feature
where you can return ""/nil from a forward/backward token function and
it means “do something sensible based on the syntax table” is actually
rather annoying to work with, both because it means forward/backward
token functions cannot be used as motion commands for testing, and
because it means sometimes you might discover, after having moved
point some distance, that you need to put it back where it
was. save-excursion helps, but there’s no way to exit a save-excursion
*without* putting point back; so you wind up with nasty back-and-forth
dances like

(defun maybe-forward-merged-token (next merged otherwise)
  (if-let*
      ((end-of-next
        (save-excursion
          (forward-comment (point-max))
          (when (re-search-forward next nil t)
            (point)))))
      (progn
        (goto-char end-of-next)
        merged)
    otherwise))

I think it would be better if the “do something sensible” behavior
were in smie-default-(forward/backward)-token, which would *always*
move over what it thinks the next token is and return it.
Language-specific token motion functions would be obliged to always
move as well, and they could wrap the default functions.]

Now, the idea.  Rust-style control flow blocks are a bad fit for OPGs
because in

while i < limit {
    operation();
    i = i + 1;
}

the opening curly brace after `i < limit` is doing double duty as the
*end* marker for the controlling expression and the *beginning* marker
for the loop body.  One would *like* to write a BNF something like

    (stmts (stmt) (stmts ";" stmt))
    (block "{" stmts "}")
    (stmt
      (expr)
      ("while" expr block)
      ; etc
    )

but that’s not allowed because you can’t have two consecutive
nonterminals in a BNF that’s to be lowered to an OPG table.  I am
fairly sure that the contorted grammar I had to write instead is a
significant chunk of why the indentation engine is doing the silly
things I described in my previous message.  (The situation with
Roto is actually even worse than it might appear from the BNF in
the roto-mode.el I sent with the previous message, because { ... }
in Roto can be notation for an “anonymous record” as well as acting
like a progn form in Lisp.  I had to omit that part of the real
grammar entirely.)

It occurred to me that we might be able to lift this limitation of
OPGs, at least in the cases where it matters for this style of
language, by leaning harder on parse-partial-sexp.  What if SMIE
partitioned the buffer into chunks by parenthesis depth, parsed
each chunk independently, and treated chunks at deeper nesting
levels than the current one as *terminal* symbols?  I’m not sure
what that would do to the existing SMIE BNF minilanguage, but it
would mean that ("while" expr BLOCK) would no longer be invalid.
We’d need a way of marking Algolesque block opener/closer keywords
as parentheses to be matched by parse-partial-sexp (probably text
properties applied by the syntax-propertize-function), but I think
that would probably be a useful thing to have *anyway*.

End of ramble, thanks for listening.
zw