Re: GCC LLM policy adopted

Andrey Rakhmatullin <[email protected]> writes:
> On Fri, Jul 31, 2026 at 08:54:05AM -0700, Russ Allbery wrote:

>> so I think it's reasonable to be somewhat more concerned in the Debian
>> context that a practice of incorrporating large blocks of LLM-generated
>> patches into packages could further exacerbate a problem Debian already
>> has: Insufficient human understanding of the changes in and
>> interactions between distribution packages to make good strategic
>> decisions.
>>
>> I'm guessing you're going to rightfully point out that we already do a
>> bad job at this, and I agree, but I still think that unsupervised
>> agentic development (to take the most extreme example) would make that
>> problem noticeably worse.

> This may be perceived as trolling but I sincerely don't think the
> typical approach of pushing stuff until CI is green and then uploading
> stuff until excuses are empty is worse than what a modern agent can do
> (and they should at least run linters locally), and I sincerely think
> that modern agents should be better than humans at understanding our
> processes, because they at least read the docs when those exist.

Yes, this is part of why I'm excepting a lot of packaging cases from my
concern about human understanding. The packaging for most packages in
Debian is largely mechanical and I'm not sure a current LLM agent is going
to be worse than a human, particularly a human who is trying to maintain a
lot of packages in a small window of volunteer time. There just isn't that
much to understand.

But I also don't want to simply say "we already have an artisinal version
of this problem that we're not really addressing; let's automate that
problem at scale." Among other things, I don't like the idea that we
should decide that it doesn't matter that our packaging processes are
complex and our tools require a lot of human hand-holding because we can
just throw that all into an LLM black box. LLM prompts are a weird type of
high-level programming language that is very powerfully abstract but
stochastically unreliable. Can't we do better? Why don't we have a
reliable implementation of the same tool?

Some days I feel like I'm part of a dying breed, but I don't like
computers that are stochastically unreliable. Part of what I like about
programming is setting up mechanisms whereby a computer will so reliably
do precisely what I want it to do, every time, without fail, that I can
stop thinking about that task entirely. If it randomly fails enough to
require human review, I can't do that, and a large part of the magic of
automation goes away, at least for me.

> (I'm not the one suggesting unsupervised agentic development! But asking
> an LLM how to handle an upload with a bumped SONAME will already produce
> better results than asking some of our library maintainers.)

I think that's a reasonable belief, but I also don't understand why we
can't have an entirely deterministic program that we tell to handle an
SONAME bump. Why is LLM prompting the best current way to automate one of
the more common tasks for a library maintainer in Debian? That gap feels
closer to the root problem.

Obviously, my preference would be for us to have human-designed and
human-written well-maintained tools and simplified processes that make our
most common tasks trivial to perform. But if I have to choose where to use
an LLM, I personally would rather see people use an LLM to help them write
the tool than to see people use an LLM to perform the task, even though
writing the tool is riskier and requires more careful review.

There are a number of different reasons for this, including auditability
of the tool once it's written. But perhaps the most obvious is that as
long as the most capable models are under the control of corporations that
are not necessarily aligned with our interests, chosing a course of
dependency on those corporations feels much riskier than using their
resources to write free tools that are independent of their continued
cooperation.

And yes, that's somewhat contradictory to my previous point that I'm more
nervous about using LLM-generated code than using LLMs for review. But I
think both positions start from the same concern: My goal is to have
software and systems that humans understand and that are structured to
help other humans analyze and maintain them.

If LLMs are useful in building such software and systems, well, I have
ethical concerns with how the current LLMs were made and are run, but I
understand why that's a complex topic and people's ethical opinions will
differ. But the world I'm the most worried about is a world in which we
have effectively lost the source code to our software and systems because
we have stopped caring about human comprehensibility and human expertise,
and they have become LLM-generated spaghetti that only other LLMs can
follow and modify.