bug#81520: Add pre-receive hook to Savannah to reject LLM-encumbered commits
Jim Porter <[email protected]> Fri, 31 Jul 2026 09:52:29 -0700
| Newsgroups | gmane.emacs.bugs |
|---|---|
| Message-ID | <[email protected]> |
On 7/31/2026 3:51 AM, Eli Zaretskii wrote: > One alternative would be to search the entire log message and > author/committer addresses for regular expressions matching > "Anthropic", "ChatGPT", "OpenAI" etc. Regexps make it easier to adapt > to changes and unanticipated issues, and can be general enough to be > relatively insensitive to minor changes in email addresses. Yeah, I think regexps would make sense here. The email addresses are still probably the most accurate indicator though. We want to disallow an address like "[email protected]", but not disallow the word "copilot" entirely. > Btw, we must spell out these restrictions in CONTRIBUTE. I don't mind writing the description for CONTRIBUTE. Most of that is separate from the exact technical means for the hook anyway; the hook is intended to be only a friendly check for honest mistakes and not a strict policy enforcement system anyway. >> I think that Jim has a good point about commit-msg vs. pre-push: >> I would be concerned about the LLM just removing the attribution in >> order to avoid the effects of the hook. Recent models are particularly >> notable for their persistence. It's somewhat reminiscent of the >> make-more-paperclips-by-any-means thought experiments. > > Sorry, I don't understand the scenario. If the attribute is removed, > how could the hook that searches attributions work in the first place? Many LLM harnesses (the programs that send/receive text to/from the underlying model) provide the ability for the LLM to run arbitrary commands. The technical details don't matter much for this discussion, but with such a harness, you could prompt the LLM that it should generate some code and commit it to a branch. In practice, this is how most LLM attributions in commit messages come about. Users calling an LLM as just a chatbot would run "git commit" themselves, and probably wouldn't include a Co-Authored-By field or anything else we could detect. The problem is that when the LLM initiates the "git commit", it will get back the result of that command. Most LLMs are designed to try different things if a command fails, including things we humans might consider malicious. If the commit-msg hook causes "git commit" to fail with a message saying we reject commits with a "Co-Authored-By: Some LLM" field, there's a good chance the LLM will remove that line and retry the commit. The user might not even realize it happened unless they look carefully at the LLM's logs. > In any case, my point is that delaying the rejection until push time > is much worse than rejecting commits, for the reasons I tried to > explain. If we must have a pre-push hook (which I'm not yet convinced > we do), then at least let's have both pre-commit and pre-push hooks. We could also do an applypatch-msg (or pre-applypatch) hook. If I'm understanding your concern Eli, this should make your life easier without running into the above problems with a commit-msg hook. This hypothetical hook would run when you call "git am foo.patch" and reject it immediately so that you don't have to remove the commit later when preparing to push.