Re: LLM usage in Debian -- ballot options comparison

Hello Lucas and others,

I've been reading this discussion, but didn't reply so far (aside from 
the message I just sent). I'd like to share my opinion as well, so here 
it is.

Let's start with the big picture: my opinion is that LLMs are an 
interesting technology, but their current effect on society is extremely 
problematic. I think regular people (i.e.: everyone except scientists) 
should avoid using them, especially for tasks for which other tools are 
available, such as translations or writing code. Even if those other 
tools are technically inferior.

When I talk about the effects on society, I agree with most points from 
Johannes, and I'll add one more below. I also want to thank Lucas for 
the respectful way that he handles this. (Side note: I am not offended 
by the summary he posted; I think it makes sense in this discussion to 
show what LLMs can do, especially because that was also being discussed.)

The "new" point (it was mentioned before, but not discussed much) is 
about human brains: the way LLMs are presenting themselves, is that they 
will do all your work. The way our brains respond to that is "let me see 
what you can do". We closely check their work for a while and once we're 
confident that they are indeed good at it, we scale down those reviews: 
they are boring and apparently unnecessary. But LLMs can do a lot of 
things right, and then mess up in ways that actual intelligence would 
not. So this leads to things breaking, unless you are extremely careful 
to avoid that, and I'm not even sure if our brains are capable of that.

What makes it worse, is that they claim that they can do the thinking 
for you. Brains are always looking for shortcuts to spend less energy, 
and thinking takes a lot of energy. So in the same way, unless we are 
very careful (and again I'm not sure if our brains can even do that in 
the long term), we are going to let them do the things that used to 
require our thinking, and we're doing it less and less ourselves. 
Apparently science has already shown that LLMs make people dumber; I 
think that is this effect. When we are very careful, as Lucas suggests, 
I think it may be possible to slow that effect. I'm not so sure if it's 
possible to entirely stop it, and it worries me a lot that so many 
people are stepping into this "trap".

Another related effect is that it removes creativity. I've heard from 
multiple sides that when people started using LLMs, all their works 
started looking the same, "soulless". Their games are no longer fun, 
their texts are no longer interesting to read. This may not be the most 
relevant for packaging (because a boring package is good), but for most 
tasks that LLMs seem to be used for, I think creativity is important and 
leaving all the "boring" parts to an LLM results in default choices for 
all of those, which is very unpleasant to experience as a player/reader/etc.

Getting back to the main discussion: In terms of this GR, I think there 
are two parts that are being discussed: a position statement and rules 
for our project.

For the position statement, we do not need to worry about enforcing 
anything. I would summarize my position, which I hope the project will 
agree with, as: The way LLMs are currently organized in the world has 
extreme costs that accelerate the destruction of the climate, both 
natural and social, that was already ongoing before LLMs appeared. This 
is extremely worrying, and I strongly recommend everyone to avoid using 
LLMs that are controlled by big players on the market.

On 8/1/26 6:50 PM, Lucas Nussbaum wrote:
 > Of course, the landscape of AI corporations is frightening. But
 > there's hope to build alternatives, with open-source agents and
 > open-weight or maybe even open-source models.

While those may be good in the future, I think that right now we should 
have a position on the current state, which is that those corporations 
are having a disastrous effect on the climate and they are pushing 
fascism. And I don't think "it may get better later, so let's jump on 
board" is a good response to the severity of the problems. It wasn't 
this bad when the internet started; that didn't literally threaten our 
ecosystem, for example. And while they were capitalists and therefore 
fascism-related, it was nowhere near the level of people like Elon Musk.

When it comes to rules for ourselves (be it by changing the social 
contract, or amending the code of conduct), I think it is good to set an 
example that follows the spirit of the above summary. I do not think the 
SC is the right document for that, as things may change in the future 
and the SC is almost impossible to change. Also I don't think we need to 
change it: I believe as it currently reads, it can already be 
interpreted to mean this. After all, turning the planet into an 
unlivable wasteland is neither good for our users, nor for free 
software. So all we need to do is explain that we agree with that 
interpretation.

Note that I do not think it makes sense to put sanctions on things we 
cannot check. LLMs were initially designed to be turing machines and 
that continues to be a large part of their focus. Saying "if you use an 
LLM, we will punish you" is sure to lead to a lot of false positives and 
false negatives. So I think the best we can do is say "you are not 
supposed to do this", and trust people to follow our rules.

Also, while I would love to live in a world again where many people 
avoid LLMs, we don't live in that world. So for practical reasons, we 
cannot set a rule that upstream work must be made without LLM 
assistance. We can still ask for it, but we must accept that not 
everyone will follow our advice. I also do not think we need to tell 
people that they cannot copy LLM-assisted work from a bug report, for 
example. I'd just treat that as upstream work. As a DD however, I do not 
think it should be acceptable to use an LLM with an upstream hat on and 
then copy that into the package with a maintainer hat on, or to ask 
someone else to use the LLM for you. I think it should be easy to 
understand what is and what isn't in the spirit of the rules I propose. 
And of course we have procedures for handling disagreements if it gets 
that far.

I feel most of the discussion (with the exception of the mail I'm 
replying to) has been people talking past each other: anti-LLM people 
have mostly argued that the climate and fascism are the big problems, 
while pro-LLM people have almost exclusively made the point that LLM 
output can technically be of high quality. Both of these can be true at 
the same time.

But to eleborate a little on that last point: they are only high quality 
if they are used properly. Lucas explained how he works with them, and 
that does sound like a good workflow (but I still think the it carries 
the risk of losing brain power in the long run). However, on the 
internet we see a lot of examples of people using them the wrong way. 
That leads to them going crazy and breaking their software, among other 
things.

 > Also,
 > both challenge teaching and evaluation practices (like calculators
 > before), but that's a pedagogy problem, not a technology problem.

This is correct, but just saying that is not good enough. If we are 
deciding that LLMs are acceptable to use (which again, I don't think we 
should), then IMO we must set up systems to train people on using them. 
Or else we are going to see a lot more broken packages.

> Also, it's worth noting that it might be a bit early to weigh costs and
> benefits of AI.

When talking about a total ban for all eternity, sure. But that doesn't 
mean we can't do anything now. The current situation is extremely 
problematic, and we are allowed to have an opinion on it (including 
"don't use them"), even if the situation (and then also our opinion) may 
change in the future.

> So, despite all the costs you listed, I think that the Free Software
> community should explore how to build freer, more independent and better
> AI, even if not everything will be super-free and perfect from day one

I agree, which is why I made an exception for science. I do not think it 
helps towards that goal at all when people are using LLMs in their 
current state to search for facts, or to translate some text. Such uses 
do still have all the downsides though.

Thanks,
Bas