Re: Gap analysis

On 2026-05-10 08:38, Jonas Hahnfeld via Discussions on LilyPond 
development wrote:
> (reflecting the current VERSION)
> 
> This is from the fear that LLMs will base their proposal on existing
> conversion rules and tests that, by definition, will exist for older
> versions... Disclaimer: I don't actually know how "good" LLMs would be
> in that regard.

Your intuition is good.  While testing 
https://gitlab.com/lilypond/lilypond/-/merge_requests/3049 last week, I 
saw my agent sometimes get into a loop trying to figure out what the 
current version was.  To avoid that, I added instructions for finding 
the version of the next release in VERSION.

My testing has been light.  I've mainly been siccing it on open reviews, 
which isn't much.  I'm trying to keep this checklist skeletal, as I 
believe that is the long-term ideal.

Over time, I (if not others) will probably add more specialized agent 
skills (e.g., lilypond-regression-testing, lilypond-engravers, 
lilypond-performers) that an agent can load when it reckons that deeper 
scrutiny is deserved.

At this early point, I'm content if an agent can identify big gaps, like 
adding a music function without adding any regression tests.  If it 
happens to notice that there are new tests with an outdated 
copied-and-pasted version, that's a bonus.
-- 
Dan