bug#81520: Add pre-receive hook to Savannah to reject LLM-encumbered commits
Jim Porter <[email protected]>
| Newsgroups | gmane.emacs.bugs |
|---|---|
| Message-ID | <[email protected]> |
On 8/18/2026 3:29 AM, Sean Whitton wrote:
>> build-aux/git-hooks/commit-msg-files.awk | 20 ++++----
>> build-aux/git-hooks/pre-push | 60 ++++++------------------
>> 2 files changed, 23 insertions(+), 57 deletions(-)
>
> I'm uncomfortable with large refactorings like this without tests to
> prove that they don't also introduce a functional change. Is it not
> possible to make the substantive changes you want to make without also
> including these refactorings? Alternatively, perhaps you could write
> some tests.
I removed the changes that aren't really necessary here. For tests, do
you have any preferences? Since the pre-push hook relies on having some
commits in your local clone that aren't upstream on Savannah yet, the
tests could be somewhat disruptive (e.g. I'd need to generate a branch
and some phony commits to check things). Alternately, I could write a
test script that initializes an all-new Git repo and runs through some
cases.
Testing commit-msg-files.awk is a lot easier, since we can run it on
historical commits. You can test it out manually with something like the
following:
git rev-list HEAD -n2500 --reverse | \
awk -v -f build-aux/git-hooks/commit-msg-files.awk
It should produce a few error messages for commits with incorrect files
listed.
>> +** LLM usage policies
[snip]
>
> The text should make it clear that it's a provisional policy pending
> work with the FSF is doing with lawyers.
Ok, I've updated this to clarify that the final policy is still up in
the air.
> I don't think we should prioritise the LLM failure like this.
> I.e. I think you should have only the latter of these if statements.
>
> I don't think we need a special .txt file here.
> The combination of CONTRIBUTE, AGENTS.md and CLAUDE.md is enough.
I removed the .txt file so we just print the final message now.
>> +function get_short_sha(commit_sha) {
>> + cmd = ("git rev-parse --short " commia_sha)
>
> Type "commia". Surprised this didn't come up in testing.
>
> Probably the refactoring to prepare a short SHA in this more principled
> way (thanks) should be its own commit.
Oops. I forgot to commit the last of the changes I had in my work tree,
so this and a couple other problems slipped through. I removed this part
and any other unnecessary changes. We can worry about them later.
>> -BEGIN {
>> - if (reason == "pre-push")
>> - verbose = 1
>> -}
>> -
>> /^[a-z0-9]{40}$/ {
>> - if (! check_commit_msg_files($0, verbose)) {
>> + if (! check_commit_msg_files($0, verbose))
>> status = 1
>> - }
>> }
>>
>> END {
>> - if (status != 0) {
>> - if (reason == "pre-push")
>> - error_msg = "Push aborted"
>> - else
>> - error_msg = "Bad commit message"
>> - printf("%s; please see the file 'CONTRIBUTE'\n", error_msg)
>> - }
>> exit status
>> }
>
> Not sure why you're making these structural changes.
Since the pre-push hook now runs multiple scripts (commit-msg-files.awk
and commit-msg-llm-check.awk), I made the pre-push file responsible for
printing the single, final error message to the user.
>> +# Check for fields that commonly indicate LLM usage.
>> +tolower($0) ~ /^(assisted-by|ai-assisted):/ { report_error($0) }
>> +tolower($0) ~ /^claude-session:/ { report_error($0) }
>> +
>> +# Check for common LLMs/harnesses. We purposefully exclude "Claude"
>> +# here because it's also a human name.
>> +tolower($0) ~ /openai/ { report_error($0) }
>> +tolower($0) ~ /anthropic/ { report_error($0) }
>> +tolower($0) ~ /copilot/ { report_error($0) }
>
> Maybe we could match 'claude' case-sensitively?
But that could hit people whose email addresses contain the substring
"claude". We do have existing contributions that would have been flagged
with a check like this, from Clément Pit-Claudel.
On 8/18/2026 4:16 AM, Eli Zaretskii wrote:
>> Date: Mon, 17 Aug 2026 13:07:30 -0700
>> Cc: [email protected]
>> From: Jim Porter <[email protected]>
>>
>> +# Check for fields that commonly indicate LLM usage.
>> +tolower($0) ~ /^(assisted-by|ai-assisted):/ { report_error($0) }
>> +tolower($0) ~ /^claude-session:/ { report_error($0) }
>> +
>> +# Check for common LLMs/harnesses. We purposefully exclude "Claude"
>> +# here because it's also a human name.
>> +tolower($0) ~ /openai/ { report_error($0) }
>> +tolower($0) ~ /anthropic/ { report_error($0) }
>> +tolower($0) ~ /copilot/ { report_error($0) }
>
> I think I'd prefer to have these in a single list, not as separate
> tests. That would make it easier to add/remove/modify the regular
> expressions as needed.
Does the attached look ok? I could make this a single regexp, but that
seemed harder to maintain if the list gets much longer...
> I also wonder how should we test these before we release them into the
> wild. Is it reasonable to write a script that will run the detection
> code on a large chunk, say the last few years, of the commit messages
> in our repository? I'm especially bothered by any false positives, of
> course, because if there are too many of them, people will disable the
> hooks.
Attached is a script you can use to run the LLM check on historical
commits (it defaults to the last 1000, but pass any number you like as a
command line arg). I've run this on the last 15,000 commits and saw no
errors. (I also locally tweaked commit-msg-llm-check.awk to flag my own
commits and the script successfully complained about them.)
0001-Simplify-pre-push-hook.patch
(text/plain, 3.2 KB)
From e87a11475afa5cc6f7f1dfd30865278f90fdbf90 Mon Sep 17 00:00:00 2001 From: Jim Porter <[email protected]> Date: Tue, 18 Aug 2026 09:37:18 -0700 Subject: [PATCH 1/2] Simplify pre-push hook This uses a single invocation of "git rev-list" instead of repeatedly calling "git branch --contains" to compute the list of commits about to be pushed, which is much more efficient. * build-aux/git-hooks/pre-push: Convert Awk script to Bash and use "git rev-list" to get all commits to be pushed. --- build-aux/git-hooks/pre-push | 60 +++++++++--------------------------- 1 file changed, 15 insertions(+), 45 deletions(-) diff --git a/build-aux/git-hooks/pre-push b/build-aux/git-hooks/pre-push index 81ba85697c3..009cab73157 100755 --- a/build-aux/git-hooks/pre-push +++ b/build-aux/git-hooks/pre-push @@ -32,6 +32,8 @@ ### Code: HOOKS_DIR=`dirname "$0"` +REMOTE_NAME="$1" +NULL_SHA="0000000000000000000000000000000000000000" # Prefer gawk if available, as it handles NUL bytes properly. if type gawk >/dev/null 2>&1; then @@ -40,49 +42,17 @@ else awk="awk" fi -# Standard input receives lines of the form: -# <local ref> SP <local sha> SP <remote ref> SP <remote sha> LF -$awk -v origin_name="$1" ' - # If the local SHA is all zeroes, ignore it. - $2 ~ /^0{40}$/ { - next - } +while read local_ref local_sha remote_ref remote_sha; do + if [ "$local_sha" = $NULL_SHA ]; then # Deleted the ref. + continue + fi - # Check any lines with a valid local SHA and whose remote ref is - # master or an emacs-NN release branch. (We want to avoid checking - # feature or scratch branches here.) - $2 ~ /^[a-z0-9]{40}$/ && $3 ~ /^refs\/heads\/(master|emacs-[0-9]+)$/ { - newref = $2 - # If the remote SHA is all zeroes, this is a new object to be - # pushed (likely a branch)... - if ($4 ~ /^0{40}$/) { - back = 0 - # ... Go backwards until we find a SHA on an origin branch. - # Stop trying after 1000 commits, just in case... - for (back = 0; back < 1000; back++) { - cmd = ("git branch -r -l '\''" origin_name "/*'\''" \ - " --contains " newref "~" back) - rv = (cmd | getline) - close(cmd) - if (rv > 0) - break; - } - - cmd = ("git rev-parse " newref "~" back) - cmd | getline oldref - if (!(oldref ~ /^[a-z0-9]{40}$/)) { - # The SHA is misformatted! Skip this line. - next - } - close(cmd) - } else if ($4 ~ /^[a-z0-9]{40}$/) { - oldref = $4 - } else { - # The SHA is misformatted! Skip this line. - next - } - - # Print every SHA after oldref, up to (and including) newref. - system("git rev-list --first-parent --reverse " oldref ".." newref) - } -' | $awk -v reason=pre-push -f "$HOOKS_DIR"/commit-msg-files.awk + # Check any lines whose remote ref is master or an emacs-NN release + # branch. (We want to avoid checking feature or scratch branches + # here.) + if echo "$remote_ref" | grep -Eq "^refs/heads/(master|emacs-[0-9]+)$"; then + # List the SHAs of all commits to be pushed to the remote for the + # current local SHA. + git rev-list "$local_sha" --not --remotes="$REMOTE_NAME" --reverse + fi +done | $awk -v reason=pre-push -f "$HOOKS_DIR"/commit-msg-files.awk -- 2.25.1
0002-Check-commits-for-LLM-usage-during-commit-msg-and-pr.patch
(text/plain, 9.7 KB)
From 0b2f0964e87ed315f93fb5d466e1aeccdee3c2c5 Mon Sep 17 00:00:00 2001 From: Jim Porter <[email protected]> Date: Tue, 18 Aug 2026 09:41:31 -0700 Subject: [PATCH 2/2] Check commits for LLM usage during commit-msg and pre-push hooks * CONTRIBUTE (LLM usage policies): New section. * build-aux/git-hooks/commit-msg-llm-check.awk: New file... * autogen.sh: ... install it. * build-aux/git-hooks/commit-msg-files.awk: Accept 'verbose' variable directly from the command line, and update callers to print final summary messages. * build-aux/git-hooks/commit-msg: Call 'commit-msg-llm-check.awk'. * build-aux/git-hooks/pre-push: Call 'commit-msg-llm-check.awk'. (get_outgoing_shas, get_commit_msg): New functions. --- CONTRIBUTE | 12 ++++ autogen.sh | 2 +- build-aux/git-hooks/commit-msg | 19 ++++-- build-aux/git-hooks/commit-msg-files.awk | 15 +---- build-aux/git-hooks/commit-msg-llm-check.awk | 67 ++++++++++++++++++++ build-aux/git-hooks/post-commit | 6 +- build-aux/git-hooks/pre-push | 43 +++++++++++-- 7 files changed, 136 insertions(+), 28 deletions(-) create mode 100644 build-aux/git-hooks/commit-msg-llm-check.awk diff --git a/CONTRIBUTE b/CONTRIBUTE index 4fdcd613d8d..ffedd631953 100644 --- a/CONTRIBUTE +++ b/CONTRIBUTE @@ -399,6 +399,18 @@ added automatically). Note that the validity checks described in the previous section are still applied, so you will have to correct any problems they uncover in the changes submitted by others. +** LLM usage policies + +The FSF is currently reviewing the legal and ethical implications of +large language models on free software. In the meantime, the GNU +project doesn't allow LLM-generated outputs in contributions. This +applies to all code, documentation, test data, and commit messages. + +This is enforced by a git hook that runs when committing or pushing +changes. This hook checks for the names of common LLMs (or their +harnesses), as well as the following fields in the commit message: +"Assisted-by:", "AI-assisted:", and "Claude-session:". + ** Branches Future development normally takes place on the master branch. diff --git a/autogen.sh b/autogen.sh index a61d74b981d..8b72a4b78cc 100755 --- a/autogen.sh +++ b/autogen.sh @@ -370,7 +370,7 @@ tailored_hooks= sample_hooks= for hook in commit-msg pre-commit prepare-commit-msg post-commit \ - pre-push commit-msg-files.awk; do + pre-push commit-msg-files.awk commit-msg-llm-check.awk; do cmp -- build-aux/git-hooks/$hook "$hooks/$hook" >/dev/null 2>&1 || tailored_hooks="$tailored_hooks $hook" done diff --git a/build-aux/git-hooks/commit-msg b/build-aux/git-hooks/commit-msg index ddde1b4e586..9bfd70cd2df 100755 --- a/build-aux/git-hooks/commit-msg +++ b/build-aux/git-hooks/commit-msg @@ -20,6 +20,8 @@ # Written by Paul Eggert. +HOOKS_DIR=`dirname "$0"` + # Prefer gawk if available, as it handles NUL bytes properly. if type gawk >/dev/null 2>&1; then awk=gawk @@ -46,7 +48,7 @@ at_sign= fi # Check the log entry. -exec $awk \ +$awk \ -v at_sign="$at_sign" \ -v cent_sign="$cent_sign" \ -v file="$1" \ @@ -176,9 +178,18 @@ at_sign= status = 1 } } - if (status != 0) { - print "Commit aborted; please see the file 'CONTRIBUTE'" - } exit status } ' <"$1" +lint_result=$? + +AUTHOR=$(git var GIT_AUTHOR_IDENT | sed -n 's/^\(.*>\).*$/\1/p') +COMMITTER=$(git var GIT_COMMITTER_IDENT | sed -n 's/^\(.*>\).*$/\1/p') +(echo "Author: $AUTHOR"; echo "Committer: $COMMITTER"; cat $1) | \ + awk -f "$HOOKS_DIR"/commit-msg-llm-check.awk +llm_result=$? + +if [ "$lint_result" -ne 0 -o "$llm_result" -ne 0 ]; then + echo "Commit aborted; please see the file 'CONTRIBUTE'" + exit 1 +fi diff --git a/build-aux/git-hooks/commit-msg-files.awk b/build-aux/git-hooks/commit-msg-files.awk index 852488b551e..31747215dfd 100644 --- a/build-aux/git-hooks/commit-msg-files.awk +++ b/build-aux/git-hooks/commit-msg-files.awk @@ -105,24 +105,11 @@ function check_commit_msg_files(commit_sha, verbose, changes, good, \ return good } -BEGIN { - if (reason == "pre-push") - verbose = 1 -} - /^[a-z0-9]{40}$/ { - if (! check_commit_msg_files($0, verbose)) { + if (! check_commit_msg_files($0, verbose)) status = 1 - } } END { - if (status != 0) { - if (reason == "pre-push") - error_msg = "Push aborted" - else - error_msg = "Bad commit message" - printf("%s; please see the file 'CONTRIBUTE'\n", error_msg) - } exit status } diff --git a/build-aux/git-hooks/commit-msg-llm-check.awk b/build-aux/git-hooks/commit-msg-llm-check.awk new file mode 100644 index 00000000000..99e4192f316 --- /dev/null +++ b/build-aux/git-hooks/commit-msg-llm-check.awk @@ -0,0 +1,67 @@ +# Check commit messages to reject LLM-attributed changes. + +# Copyright 2026 Free Software Foundation, Inc. + +# This file is part of GNU Emacs. + +# GNU Emacs is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. + +# GNU Emacs is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. + +# You should have received a copy of the GNU General Public License +# along with GNU Emacs. If not, see <https://www.gnu.org/licenses/>. + +### Commentary: + +# This script checks a commit's metadata for signs of LLM usage. It +# accepts a commit message with author/committer in the following form: +# +# Author: Author Name <[email protected]> +# Committer: Committer Name <[email protected]> +# Commit message +# +# It will check this info for the names of common LLMs/harnesses, as +# well as for common fields that indicate LLM usage: "Assisted-by:", +# "AI-assisted:", or "Claude-session:". + +### Code: + +BEGIN { + status = 0 +} + +/^#/ { next } # Skip comment lines + +# Check for fields that commonly indicate LLM usage. +tolower($0) ~ /^(assisted-by|ai-assisted):/ || +tolower($0) ~ /^claude-session:/ || +# Check for common LLMs/harnesses. We purposefully exclude "Claude" +# here because it's also a human name. +tolower($0) ~ /openai/ || +tolower($0) ~ /anthropic/ || +tolower($0) ~ /copilot/ { + + if (status == 0) { + if (commit_sha) + printf("LLM attributions detected in commit %s:\n", commit_sha) + else + printf("LLM attributions detected:\n") + } + status = 1 + + if (NR <= 2) # Author or Committer + printf(" %s\n", $0) + else + printf(" Line %d: %s\n", NR-2, $0) + +} + +END { + exit status +} diff --git a/build-aux/git-hooks/post-commit b/build-aux/git-hooks/post-commit index 73fd5b28780..cc02d4df719 100755 --- a/build-aux/git-hooks/post-commit +++ b/build-aux/git-hooks/post-commit @@ -43,5 +43,7 @@ else awk="awk" fi -git rev-parse HEAD | $awk -v reason=post-commit \ - -f "$HOOKS_DIR"/commit-msg-files.awk +if ! git rev-parse HEAD | $awk -f "$HOOKS_DIR"/commit-msg-files.awk; then + # It's too late to abort the commit, so just print a warning. + echo "Bad commit message; please see the file 'CONTRIBUTE'" +fi diff --git a/build-aux/git-hooks/pre-push b/build-aux/git-hooks/pre-push index 009cab73157..ff9b5e0a246 100755 --- a/build-aux/git-hooks/pre-push +++ b/build-aux/git-hooks/pre-push @@ -35,6 +35,17 @@ HOOKS_DIR=`dirname "$0"` REMOTE_NAME="$1" NULL_SHA="0000000000000000000000000000000000000000" +# List the SHAs of all commits to be pushed to the remote for the +# specified SHA. +get_outgoing_shas() { + git rev-list "$1" --not --remotes="$REMOTE_NAME" --reverse +} + +# Print the author, committer, and commit message for the specified SHA. +get_commit_msg() { + git show -s --format='Author: %an <%ae>%nCommitter: %cn <%ce>%n%B' "$1" +} + # Prefer gawk if available, as it handles NUL bytes properly. if type gawk >/dev/null 2>&1; then awk="gawk" @@ -47,12 +58,30 @@ while read local_ref local_sha remote_ref remote_sha; do continue fi - # Check any lines whose remote ref is master or an emacs-NN release - # branch. (We want to avoid checking feature or scratch branches - # here.) + # Check the file names in the commit messages being pushed to master + # or an emacs-NN release branch. (We want to avoid checking feature + # or scratch branches here.) if echo "$remote_ref" | grep -Eq "^refs/heads/(master|emacs-[0-9]+)$"; then - # List the SHAs of all commits to be pushed to the remote for the - # current local SHA. - git rev-list "$local_sha" --not --remotes="$REMOTE_NAME" --reverse + get_outgoing_shas "$local_sha" | \ + $awk -v verbose=1 -f "$HOOKS_DIR"/commit-msg-files.awk fi -done | $awk -v reason=pre-push -f "$HOOKS_DIR"/commit-msg-files.awk + files_result=$? + + # Check for LLM attributions in all outgoing commits. We do this in a + # separate pass from above so that the output is kept separate. + get_outgoing_shas "$local_sha" | (while read outgoing_sha; do + short_sha=$(git rev-parse --short "$outgoing_sha") + get_commit_msg "$outgoing_sha" | \ + $awk -v commit_sha="$short_sha" -f "$HOOKS_DIR"/commit-msg-llm-check.awk + status=$(expr "$?" "|" "$status") + # Since commands in pipelines all run in their own subshells, + # explicitly exit with the overall status so the parent shell knows + # the result. + done; exit "$status") + llm_result=$? +done + +if [ "$files_result" -ne 0 -o "$llm_result" -ne 0 ]; then + echo "Push aborted; please see the file 'CONTRIBUTE'\n" + exit 1 +fi -- 2.25.1
historical-llm-check.sh
(text/plain, 671 B)
#!/bin/sh
HOOKS_DIR="build-aux/git-hooks"
# Print the author, committer, and commit message for the specified SHA.
get_commit_msg() {
git show -s --format='Author: %an <%ae>%nCommitter: %cn <%ce>%n%B' "$1"
}
git rev-list HEAD -n"${1:-1000}" --reverse | (while read outgoing_sha; do
short_sha=$(git rev-parse --short "$outgoing_sha")
get_commit_msg "$outgoing_sha" | \
awk -v commit_sha="$short_sha" -f "$HOOKS_DIR"/commit-msg-llm-check.awk
status=$(expr "$?" "|" "$status")
# Since commands in pipelines all run in their own subshells,
# explicitly exit with the overall status so the parent shell knows
# the result.
done; exit "$status")