Re: [MAINTAINERS SUMMIT] Other LLM-related topics - tags, newcomers, etc

On 26/07/17 04:54PM, Johannes Weiner wrote:
> On Thu, Jul 16, 2026 at 04:23:19PM -0400, Liam R. Howlett wrote:
> > So on Greg's note that having an LLM tag is a signal,  I question the
> > SNR now and if the signal is the same indicator across all patches.
> 
> Yes. The possible range of involvement is massive. The question I
> would find more interesting than a binary flag is who was doing what
> part of the thinking at each step of creating the patch(es).
> 
> You can use agents to draft code, make tests, run experiments, analyze
> data, and review for bugs. The human can apply a high level of
> direction, scrutiny and steering on each of these steps. Or they can
> just not do any of that and still get some kind of result.
> 
> Once a workable direction is found, the final result that gets sent to
> the list could be a clean-room implementation written by the human, it
> could be fully written by an agent, or anything in between.
> 
> And finally the human could have reviewed the work end-to-end,
> line-by-line; delegated it all to a review agent; or neither.
> 
> The process behind a patch with this tag could be anywhere on this
> spectrum. And reviewer interpretation of the tag is just as varied.
> 
> It poorly communicates what actually occurred. And there is clearly a
> stigma attached to it at this point, with the expected consequences,
> which further distorts the signal. The latter part might go away if
> there was a better protocol to convey the development process.
> 
> But I'm not sure it's practical to even try to capture all these
> nuances and precise division of labor questions in commit metadata...

Yeah, well said.

And at that point, why even have a tag?  Maybe something in the cover
letter would be better to explain what went on, but this is hardly what
and why things changed.  And without a standard way of communicating how
the tool was used, then we're back to a signal/noise issue again.

Which, I think is covered by your statement of the 'commit metadata' not
being practical for the task.

To your point of stigma being attached to patches, this is probably why
some people insist LLMs were not used when they clearly were.

Thanks,
Liam