Re: A quick thought on AI-generated bulk PRs

Hi Elliotte,

To provide some context on what is happening across the repositories, I
looked into my dashboard metrics as of this morning. We currently have 32
open Dependabot PRs across 12 repositories and *510 human-authored PRs
across 70 repositories.*

Detailed Dashboard Analysis

Dependabot Queue (32 PRs)
The overwhelming majority (20 of 32, ~63%) are low-risk CI/tooling or
patch-level bumps, such as GitHub Actions version pins and small library
point releases. These are prime candidates for batch-merging as they are
mechanical and already green.

  - Build Status: 26 are green, 4 are failing (notably #691, #749, #1694,
and #3384), and 2 show "UNKNOWN" status.
  - Age: Most were opened between 2026-08-09 and 2026-08-11. Three older
PRs stand out, including a plexus-utils bump (#3384) that has been failing
for 6 weeks.
  - Scrutiny Needed: A set of 5 PRs are major version bumps (e.g., guice,
provisio-maven-plugin) that deserve closer inspection due to potential
breaking changes.

Human-Authored PRs (510 PRs)
This backlog is a very different and clusters into a few recurring themes.

  - Categories:
      - Small Fixes and Docs: A large portion consists of small bug fixes
and documentation updates, primarily from you and me. These are generally
well-scoped and excellent candidates for fast-track review.
      - Core & Migration Work: Efforts regarding Maven 4 migration and
performance optimizations from Guillaume and Tamas have broader
architectural impacts.
      - Drafts and WIPs: There are 132 drafts (~26%), including various
experiments and proofs-of-concept that might be better tracked as design
discussions.
  - Build Status: Only ~42% (214 PRs) have a green status. A striking 45%
(229 PRs) show "UNKNOWN" status, which is especially common on older
entries where CI never ran or isn't configured.
  - Age: The backlog spans over a decade, with some PRs dating back to
2014-12-22. More than half of these open PRs predate 2025, with heavy
concentrations of aging code in surefire, shade-plugin, and core.

Overall Advice

Regarding the human-authored backlog, I suggest a multi-step approach:

1.  Fast-track the small, green fix and doc PRs to reduce the count quickly.
2.  Focus review on the Maven 4 and performance PRs.
3.  Convert experiment/WIP drafts into design discussions.
4.  Perform a stale-PR sweep on the multi-year-old PRs with "UNKNOWN"
status, particularly those from 2017–2020.
5.  Investigate potential systemic CI issues in surefire and shade-plugin
given their high failure rates.

It seems the velocity has indeed increased with targeted cleanups and
automated updates, but triaging the stale queue and fast-tracking the
low-risk, green PRs should help make the backlog manageable.

Best regards,
Sylwester


On Tue, Aug 11, 2026 at 11:35 PM Elliotte Rusty Harold <[email protected]>
wrote:

> I'm unsure what you're seeing. I'm not aware of any cases of "Run a
> prompt in "yolo" mode to make massive changes across a bunch
> of repos all at once." I think we might get there, but we're not there
> yet, or maybe someone is sending PRs I'm not seeing? Can you point to
> such a thing? There are instances of people sending out PRs one at a
> time across different repos to do something similar that needs doing.
>  Sylwester has been converting docs from apt to markdown in multiple
> repos, and I've been using LLMs to make our docs actually legible. BUt
> this is happening one repo at a time. However, most LLM assisted PRs
> focus on fixing something idiosyncratic to one plugin or library, not
> something that's broken across all repos.
>
> We have had long-standing issues with failure to review PRs in a
> timely fashion. And the recent increase in LLM-assisted PRs which has
> increased code velocity has made this failing more obvious. But the
> problem is not that we are getting more valuable PRs that fix bugs and
> make improvements we've needed for years. The problem is that we're
> not reviewing them. Guillaume has, I think. set up a bot to
> auto-review PRs with Clause. At least I don't know how to otherwise
> explain the speed with which his reviews are coming lately. Perhaps he
> can confirm? In any case, that is helpful.
>
> I think we long ago passed the point where GitHub emails were useful,
> even before LLM PRs became a thing. I suggest investigating dashboards
> that show activity and PR status across the project. Right now the
> biggest hole is simply failing to notice that a PR needs review
> because it's in a plugin I haven't looked at lately.
>
> On Mon, Aug 10, 2026 at 2:29 PM Olivier Lamy <[email protected]> wrote:
> >
> > Hi there,
> > Just to be clear upfront: I’m totally pro-AI tools and use them myself.
> > But I’ve noticed a pattern lately with bulk changes that I think we
> > need to tweak.
> > Right now, the workflow looks a bit like this:
> > 1. Run a prompt in "yolo" mode to make massive changes across a bunch
> > of repos all at once.
> > 2. Merge the PRs almost immediately.
> > 3. Realise the initial prompt missed something, which triggers another
> > massive batch of PRs to fix the first batch.
> > 4. back to 1.
> >
> > The main issue here is notification fatigue. We’re getting blasted
> > with so many emails and alerts that it’s impossible to keep up. When
> > that happens, we tune out the noise and stop reviewing entirely, which
> > is exactly when real issues slip through the cracks.
> >
> > Since AI is incredibly useful for these kinds of bulk updates, I think
> > we can do this more effectively/wizely.
> > Could we try this approach instead?
> >
> > Start small: Run the prompt and generate a PR for just one repo first.
> > Review: Have the community conduct a thorough, in-depth review of that
> > single AI-generated PR.
> > Refine: Tweak the prompt based on the feedback. Once it's solid, we
> > can save it as a reusable "skill" that the community trusts.
> >
> > Scale up: Once the single repo is good to go, then roll the prompt out
> > to the rest of the repos.
> >
> > I think slowing down just a little bit at the start will cut down on
> > the noise, save us a lot of context-switching, and make us all way
> > more confident in the code we're merging.
> >
> > Thoughts?
> >
> > Thanks,
> > Olivier
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]
> >
>
>
> --
> Elliotte Rusty Harold
> [email protected]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>