Re: Stop false review statements

On Mon May 18, 2026 at 7:19 PM CEST, Roman Gushchin wrote:
>
>> On May 17, 2026, at 8:56=E2=80=AFAM, Danilo Krummrich <[email protected]> =
wrote:
>> That said, I personally don't mind too much, I really like sashiko, whic=
h is
>> also why I asked for adding the driver-core list. My experience has been=
 that it
>> does a very decent job in providing feedback for C code; my feeling is t=
hat
>> feedback for Rust code is not quite on par yet, but of course it also hi=
ghly
>> depends on the complexity and scope of the corresponding changes.
>
> This is super interesting. An obvious idea is that the training set is
> relatively limited, if we=E2=80=99re talking rust for kernel code. Did yo=
u notice any
> common topics or patterns?

My experience is - and that's probably not surprising as it may tie back to=
 the
relatively limited training set you mention above - that it sometimes loses
context at the FFI boundary, where it has to consider the semantics of the =
C and
the Rust APIs.

I think the difficult part is not necessarily that there is a language boun=
dary
by itself, but rather that it can be difficult for LLMs to capture the sema=
ntic
layer that a Rust API puts on top of a C API in order to derive a safe API.

This also has been my experience in other LLM contexts; the most significan=
t
improvements I saw in output quality have been by providing more semantic
context for the abstraction design.

Probably easy to look up, but does Sashiko consider cover letters already?

> Does it produce more false positives or worse in finding actual bugs in
> comparison to the c code?

I'd say both, and I think the reason is probably the same for both. But to =
be
fair, this is probably also biased by the class of changes I deal with on t=
he C
and on the Rust side too.

It is also fair to say that in Rust we (almost) don't have whole classes of=
 bugs
that we have on the C side; and the class of memory safety issues makes a h=
uge
proportion of bugs on the C side.

If the model is particularly good at finding memory safety issues (and I th=
ink
it is), it of course also messes with the statistics of false positives
comparing C with Rust.

> I personally think that it=E2=80=99s always better to cc some mailing lis=
t and/or
> maintainers, so there is a second pair of eyes. I totally agree that repl=
ying
> just to the author is less effective.  Of course, we can add the text you=
=E2=80=99re
> proposing, but why not simply configure sashiko to cc the mailing list?

I'm not sure it addresses much of this concern, as I prefer to keep everyon=
e the
patch (series) has been sent to explicitly in the loop.

Of course everyone has a different workflow, but I think it is fair to assu=
me
that most people who are addressed directly follow along through their inbo=
x and
not through the mailing list.

So, I think the real alternative is "reply all", which for multiple reasons=
 I'm
not convinced is the right general call to make yet.

(I do consider it for one of the components I maintain though.)

Thanks,
Danilo