Re: Thoughts about LLM usage

Hi,

On 8/9/26 4:12 AM, Andreas Tille wrote:

> For some years AI has been changing the world.  It will do so with or
> without Debian.  We can decide to ignore this change.

We're specifically not ignoring it, but asking whether this change is 
worthwhile. Part of the project says "no, for 
environmental/ethical/political reasons", part of the project says "no, 
for technical reasons", and part says "yes, for technical reasons."

I don't care much about the two factions arguing about the technological 
merits, this will sort itself out on its own, by projects failing and 
being abandoned. I'm sad about the contributor effort being lost to 
this, but this is not something I have a lot of influence on.

Since there is no significant "yes, for environmental/ethical/political 
reasons" faction, my critique focuses on technical aspects, and how even 
if you could ignore all the other bits, it would not deliver the desired 
results. I still think that environmental, ethical and political reasons 
are why we should reject the use of this technology, and any technical 
aspects are secondary.

My expectation is that it will be mostly the projects embracing LLMs 
that will fail in the longer term, precisely because it allows small 
groups of people to build really complex contraptions, or single people 
to make drive-by contributions to complex projects, and while this looks 
beneficial in the short term, it is not sustainable for a project long term.

I've had a bunch of my projects collapse under their own weight even 
without computer assistance, so the main (technical/management) result 
of LLM use is higher velocity of that failure mode, and more replacement 
projects created as a result, each causing churn and eroding trust for 
users. These projects also tend to be more insular, less community-oriented.

The other technical failure mode I can see is the exact opposite: it 
allows a non-ideal state to persist longer, by making workarounds more 
available. Basically, if we had had LLMs twenty years ago, there would 
have been little reason to develop debhelper, as we would have had a 
sufficiently low-effort mechanism to generate and modify the 
debian/rules "boilerplate" code.

Likewise, if there is a need to package sixty Rust crates, then the 
solution isn't to generate sixty similar variants of a Rust packaging 
recipe, but "dh --with-rust" and maybe a dh_make template. If the crates 
provide a useful description, "Description: ${rust:Description}" is a 
better solution than copying, with or without running it through an LLM.

The often-mentioned example of the debian/copyright file is another 
instance of this:

LLMs can provide a "good enough" file that passes cursory inspection, 
but a proper review by the maintainer takes more time than writing it by 
hand because it *still* requires all the information to be collected in 
a way that does not invent or omit copyright holders, and then the 
gathered information needs to be compared against the existing file.

It just appears to be easier to update the file using an LLM if the goal 
isn't correctness, but (antagonistically) passing the gatekeepers of the 
DFSG team. If the DFSG team replies with a list of required changes, one 
could even outsource the entire LLM review process to them.

So the technical/organizational argument against LLM use is that their 
usefulness is mainly in tasks that are organized in a suboptimal way but 
where people do not believe they have the agency to change the process 
and therefore need a way to work around it — but in turn, people getting 
used to working around suboptimal processes causes them to become 
entrenched in the first place.

I also think that the perceived usefulness of LLMs is mostly a "big 
picture" thing, as they can be applied to many different problems, and 
if they turn out to be unsuitable for a problem, then that still leaves 
all the other use cases — and that remains true even if we find them to 
be a bad match for many problems, because for each user there is always 
a fuzzily defined larger set of problems they haven't failed at, usually 
problems where the user is not a subject matter expert.

This perception is similar to how you could buy a newspaper that reports 
on politics, economics and technology and be outraged that the 
technology section is full of false information that could have been 
avoided by doing five minutes of research, but at least their politics 
and economy sections are well-informed — and then you talk to your 
pol-sci and econ friends, and they are full of rage about how their 
sections are entirely bullshit, but praise the technology section.

If we exclude any uses where the output does not pass expert review, not 
much is left, which makes it even more difficult to argue that the 
negative effects are somehow worth it — a position for which you'd first 
have to take the position that the creation of software somehow can be a 
more worthwhile goal than conserving a livable environment, a stable 
society and a healthy community, or at least the position that these 
concerns can be separated.

This is fractally wrong:

1. none of us is privileged enough to escape the consequences of 
damaging our ecological or social environment
2. even if some of us were, the creation of software is not worth it
3. the creation of software is not a goal in itself, but a means to an end
4. we won't even get good software out of it.

    Simon