Re: LLM usage in Debian -- ballot options comparison

Hi Lucas and others,

Lucas Nussbaum <[email protected]> schreef op 3 augustus 2026 14:08:40 CEST:
 >Hi Bas,

First of all, it worries me that mails which say 'climate change is the 
primary issue, but also X and Y' get responses about X and Y, but not 
about climate change.

IMO you (Lucas) make a fairly convincing point that LLMs are more than 
'planet-burning slop extractors': if careful, I believe you that it is 
possible to create non-slop. However, I have seen no serious attempt (by 
you or others) to claim that they are not planet-burners.

I don't understand how anyone can agree that LLMs destroy our ecosystem 
and not feel like that is a deal-breaker.

The only response I've seen to this point was that other activities are 
also bad (some even worse) for the climate. How is this even an 
argument? The planet is burning, we should try to stop that as much and 
as fast as we can! If others aren't stopping, isn't that even more 
reason to compensate for their irresponsible actions?

And while I'm on that point: All of our users live on this planet, so 
protecting that planet is a matter of life and death for them. In other 
words, caring about something as big as climate change during our Debian 
work is not (as was suggested elsewhere in the discussion) a violation 
of the social contract. On the contrary, I would argue that the social 
contract requires us to care about climate change and prevent it as much 
as we can.

Anyway, while climate change is by far the most important point IMO, 
I'll also reply to your answer:

 >On 02/08/26 at 09:28 +0200, Bas Wijnen wrote:
 >> The "new" point (it was mentioned before, but not discussed much) is 
about
 >> human brains: the way LLMs are presenting themselves, is that they 
will do
 >> all your work. The way our brains respond to that is "let me see 
what you
 >> can do". We closely check their work for a while and once we're 
confident
 >> that they are indeed good at it, we scale down those reviews: they are
 >> boring and apparently unnecessary. But LLMs can do a lot of things 
right,
 >> and then mess up in ways that actual intelligence would not. So this 
leads
 >> to things breaking, unless you are extremely careful to avoid that, 
and I'm
 >> not even sure if our brains are capable of that.
 >>
 >> What makes it worse, is that they claim that they can do the 
thinking for
 >> you. Brains are always looking for shortcuts to spend less energy, and
 >> thinking takes a lot of energy. So in the same way, unless we are very
 >> careful (and again I'm not sure if our brains can even do that in 
the long
 >> term), we are going to let them do the things that used to require our
 >> thinking, and we're doing it less and less ourselves. Apparently 
science has
 >> already shown that LLMs make people dumber; I think that is this effect.
 >> When we are very careful, as Lucas suggests, I think it may be 
possible to
 >> slow that effect. I'm not so sure if it's possible to entirely stop 
it, and
 >> it worries me a lot that so many people are stepping into this "trap".
 >
 >Some comments about this:
 >
 >I see the brain analogy as mostly PR (originally from Anthropic, I
 >think).

I'm not sure what analogy you're talking about; I'm talking about actual 
human brains and how they react to using LLMs.

 >At some point, some people described LLMs as "super-fast interns", then
 >as "assistants". But I find that the analogy that works best is the one
 >of the exoskeleton: it's a tool that extends the user's powers, but the
 >user still decides what it does (unless they decide not to decide on
 >some details, which is essentially what they do when vibe-coding).

I agree with this. My point is that our brains tend to prefer to limit 
their own thinking. Because of this, I expect many people to jump 
straight to vibe-coding and others to move in that direction over time.

 >I'm not sure that LLMs make people dumber. My feeling is that the
 >problem is that LLMs make dumb/simple/automatable activities obvious. As
 >a consequence, when people try LLMs, first there's excitement at being
 >able to automate some of one's activities, but then there's Fear Of
 >Becoming Obsolete: if some activities can be performed just as well by
 >LLMs, what essence does one's job retain?

I think this fear exists for some people, but I'm not one of them. My 
issue is not with automation, I love that part. My issue is that it 
doesn't work well. You described before how much effort it takes to get 
the LLM to output good code. Non-LLM-automation is not as powerful, but 
you also don't need to keep checking its output every time. For LLMs you 
do, and most peope are not going to (at some point, even if they did 
initially).

 >Of course, the fact that more activities can be automated transforms
 >jobs. The future of software engineering is probably more about
 >engineering than about coding craftsmanship: coding can be largely
 >automated, so what remains are questions about general strategy,
 >technological choices, specification and consistency with requirements,
 >design, architecture, test, verification, codebase consistency, etc.

That's how it should be, but I disagree that LLMs are that road. Again, 
you need to keep checking whether LLMs didn't hallucinate. So instead of 
spending all your time on the high level tasks, you have to spend it on 
reviewing code which is mostly correct (but the mistakes can be in 
unexpected places). And if you find a problem, you don't even get the 
satisfaction of learning something new (or seeing someone else learn 
something new). So it's very boring and deeply unsatisfying.

 >There are two consequences to this:
 >1/ Some people are attached to doing manually what can now be automated
 >(similarly to manually repairing your car even if it's more time/cost
 >efficient to have a professional do it for you). I think that's part of
 >Johannes' "taking my hobby away" argument. As a volunteer-based
 >organization, it of course makes sense for Debian to respect one's
 >willingness to perform tasks either the traditional way, or the
 >modern/efficient ways.

I would agree with this, if I would not have climate objections and if I 
would agree that it would result in high quality work (as a rule, not as 
an exception). Neither of those are true, though.

 >2/ Mental fatigue: as the "easy" tasks are now automated, what remains
 >are mostly mentally-challenging tasks. I don't think that's an issue for
 >Debian as most contributors only spend a few hours per day or week on
 >Debian stuff, but it is clearly a problem in professional environments.

I think this is a capitalist/exploitation issue. If this is handled 
properly, a full time job is fewer hours for that kind of work. (Not 
that the CEOs of large corporations are going to agree to that...)

For volunteer work, as we do in Debian, it certainly isn't a problem to 
just work fewer hours if we still get the same or more work done.

 >> Another related effect is that it removes creativity. I've heard from
 >> multiple sides that when people started using LLMs, all their works 
started
 >> looking the same, "soulless". Their games are no longer fun, their 
texts are
 >> no longer interesting to read. This may not be the most relevant for
 >> packaging (because a boring package is good), but for most tasks 
that LLMs
 >> seem to be used for, I think creativity is important and leaving all the
 >> "boring" parts to an LLM results in default choices for all of 
those, which
 >> is very unpleasant to experience as a player/reader/etc.
 >
 >I think it's useful to distinguish between true creativity, and what I
 >would call "industrial creativity". Coding is often an "industrial
 >creativity" task, where one re-does stuff that has already been done
 >elsewhere, with some adaptations (to context, or change of programming
 >language, etc.).

My point is that even things that feel industrial still affect the 
result in ways that you may not expect. Even if you don't realize it, 
you make lots of choices during 'industrial' work.

What makes it worse, is that LLM users typically think the result is 
better than it actually is; they believe the bot at least a little when 
it praises them: it has been shown that when people use an LLM, they are 
more confident (and thus less likely to ask for help), but the quality 
of their work is lower.

And some believe it when it tells them they're the new messiah, of course.

Thanks,
Bas