Re: Thoughts about LLM usage

On Wed, Aug 12, 2026 at 08:52:21PM -0500, Simon Richter wrote:
> 
> With "contributor effort", I mean the work on projects that fail, and the
> work that is required to deal with the fallout of failed projects.

There is an assumption that the use of LLM's will cause projects to
fail.  This is still to be seen, and we probably won't know this for a
while, since projects fail for many reasons, and if we look at the
huge numbers of abandoned repositories on github or sourceforge,
arguably more FOSS projects fail than suceed.

We could also make the case that without the use of LLM's, projects
will be more subject to fail, given that the number of AI-generated
security vulnerabilities are going to up whether a project takes a
King Canute approach to LLM's or not.

> The flip side of the increased effectiveness is that it often becomes easier
> to rebuild a project from scratch instead of contributing to an existing
> project. Typically the main feature of the rewrite is an extensive test
> suite "to make it easy to contribute", in the hope of attracting more people
> — but the same mechanism that made it feasible to rewrite the project in the
> first place also makes it unattractive to contribute to it.

There are examples where experienced maintainers have been using LLM's
to create or improve test suites, or to create new unit tests.  A
classic example of this is rsync, where increasing featurities made it
harder and harder to make changes (including urgently needed security
fixes) without breaking corner-case features.  So Tridge's use of
LLM's was to make it easier to make changes, and was not the cause of
the regressions.

I have started using LLM's to improve my file system test appliance.
Some of those changes have the goal to make to automatically detect
whether the backport of fixes to stable kernels might or might cause
test regressions (and bug escapes that could cause production systems
to crash).

Ext4 has been around for two decades, and I have no real worries that
improving a test suite will make it possible for someone to rewrite
the file system.  If someone wants to think so, they are welcome to
try...  :-)


I have seen an increased number of new contributors to ext4, and some
of it have involved AI assisted changes.  Some of those patches are
fixing real bugs (that were discovered using AI; in some cases, due to
automated AI review of earlier changes).

It's still too early to see how many of these new contributors will
turn into long-term, experienced contributors, but my experience is
that building a community has always been a pipeline question, and so
increasing the number of first-time contributors is a good thing (tm).
There has always been some number of contributors that will only
submit a single change before disappearing forever, both before and
after the advent of AI.  But some small percentage --- say, 5% or 10
--- stick around.

It does appear that for some of these new contributors, having AI
assistance and AI-automated review provides a more gentle on-ramp, and
it appears that some have been more engaged as a result of the AI
review as opposed to being scared away.  Let's see how things work out
in a few years, but to date I'm cautiously optimistic.

Cheers,

						- Ted