Re: Full subsystem codebase scan for identifying pre-existing issues?

Hi Derek,

On Wed, Jul 29, 2026 at 6:43=E2=80=AFPM Derek Barbosa <[email protected]>=
 wrote:
>
> Hi,
>
> On Wed, Jul 29, 2026 at 01:34:10PM -0700, Joanne Koong wrote:
> > Hi,
> >
> > Is there a way for Sashiko to run a full scan of a subsystem/tree to
> > identify all the pre-existing issues so that they can be fixed
> > proactively?
>
> At the moment, no. At least nothing that I am aware of :)
>
> There have been some small conversations about such functionality, but no=
thing
> has really taken shape yet. One of the concerns was trying to validate th=
e sheer
> number of potential-false-positives, and how to present such information =
in an

Gotcha, thanks for the context :) That makes sense, that sounds tough
balancing potential false positives.

Just thinking out loud here... if those false positives are currently
getting reported/flagged by Sashiko on contributor submissions
anyways, maybe it'd still be helpful for them to show up in the
subsystem codebase scan so maintainers can course-correct the rules
for them in Sashiko (and hopefully save contributors not as familiar
with the subsystem some confusion :)).

> actionable manner. IIRC syzbot/syzkaller's way of storing such data was u=
sed as
> a comparison point here.
>
> FWIW, Chris Mason has a repository for a tool that runs such scans on cod=
ebases
> -- paired with tools like semcode. His prompts are the same ones that we
> frequently sync back to Sashiko. It won't solve the other point you made,=
 but it

Awesome! I'm a huge fan of Chris's work. I wasn't sure if there were
differences in the analysis between Sashiko and Chris's tools, but
it's useful to know that the prompts are synced from Chris's repo.
Anecdotally, when I ran a couple of my series through Chris's AI
systems a couple months ago, there were some things it pointed out
that Sashiko didn't and vice versa, but I think that's probably more
due to the underlying LLM being used (eg claude vs gemini).

> may be worth looking into. [0][1]
>
> > Currently, they're flagged on patch reviews, which can be
> > a bit frustrating for contributors who didn't introduce the problem,
> > and adds a bit of noise for maintainers/reviewers looking at the
> > Sashiko report.
>
> Unfortunately, you aren't the first to report a problem with these.
>
> Currently, in the 11th stage instruction in the code, pre-existing issues=
 are
> propagated in the report like so:
>
> <snip>
>
> CRITICAL RULE: If a finding is flagged as pre-existing (`\"preexisting\":
> true`), you MUST explicitly state in your inline comment that this issue =
is
> pre-existing and was not introduced by the patch under review. Use phrasi=
ng like
> \"This isn't a bug introduced by this patch, but...\" or \"This is a
> pre-existing issue, but...\" to start the comment.
>
> </snip>
>
> Instead of them replying inline with "This isn't a bug introduced by this=
 patch,
> but by..." or "This is a pre-existing issue, but does..." how would you l=
ike to
> see this information presented/conveyed?

I (like Andrew) also find the pre-existing issue reporting really
useful and definitely wouldn't want to lose them. Maybe something that
could help is if they were marked even more obviously as pre-existing
issues? (eg prefaced with a bold "PRE-EXISTING ISSUE - " or displayed
in a different font color or enclosed within parentheses). I usually
look at the web ui and sometimes when glancing at a series (eg [1]),
it would be faster to differentiate between bugs I caused vs
pre-existing ones if they were visually marked differently. This is
not a big deal at all though. Thanks for your work on Sashiko!

Thanks,
Joanne

[1] https://sashiko.dev/#/patchset/20260715174305.336261-1-joannelkoong%40g=
mail.com