Re: Meeting proposal for KernelCI Bisection

--sRUg/N4PsNSAFS4u
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Wed, Mar 11, 2026 at 10:01:08PM +0100, Guillaume Tucker wrote:

> Stage 1: the basics
> -------------------

> The first step is to implement the same kind of bisection as was
> initially done in Jenkins but this time with Maestro:

> * only for LAVA test regressions
> * triggered once all the results for a particular revision are in
> * removing duplicates with simple criteria
> * optimising builder usage while waiting for runtime test results

> The pipeline logic boils down to this:

> 1. look for base 'good' commit with reference branch (i.e. mainline)
> 2. check bad / good commits
> 3. iterate with git bisect
> 4. check found commit and revert in place to check it fixes the issue
> 5. send email report in reply to the original patch thread
>    (or alternatively build a list of recipients from the commit)

> It would be wise to send reports to a fixed audience initially for
> moderation until it's deemed good enough for automated replies.

I think there's a bunch of stuff we can do a lot better here than the v1
stuff did - the big one is sharing the builds and test runs between
multiple bisections.  Especially with the builds it's relatively common
to be able to share at least the first step or two of a set of
bisections (eg, if two tests fail on the same build we can reuse the
build), and it's not uncommon for multiple tests in the same test job
fail at once with the same or closely related causes.  You can get
something a whole lot more responsive that's still very simple in
implementation.

Doing builds and tests as Maestro jobs probably gets us a lot of the way
there, what Arisu has implemented already in kci-dev but rearranged a
bit with more of a server focus (eg, driving multiple bisects in
parallel).

> Stage 2: expanded coverage
> --------------------------

> * add primitive support for metrics-based results e.g. thresholds

BTW throwing https://otava.apache.org/ out there in case people haven't
seen it.  More relevantly there's also the fastpath bisection stuff.

> Stage 3: advanced features
> --------------------------

> Using 'git bisect' has its own limitations.  It's great during
> development but automated systems have different use cases.  A more

There's also stuff like mining history and throwing test results that we
already have in the range at the bisect (eg, if we've tested commits on
a different branch or as part of another bisect), not particularly smart
in terms of the bisect algorithms or anything but making use of the
data set that KernelCI is building up to try to improve their performance.

Another idea that's getting more into trying to be actually clever would
be looking at the builds we've already got and testing those if they're
somehow close enough to what we might have otherwise chosen rather than
spending time doing another build.

> their tree (e.g. -rc2 tag).  Semi-automated bisections is also
> something to explore, with a command line that 'git bisect' can run
> on each iteration and the ability to trigger fully automated ones
> manually.

There's the bisect stuff Arisu did in kci-dev that I mentioned for the
first bit.

> So this is a bit of a brainstom or braindump exercise without
> actually diving into anything in much detail.  It'll be interesting
> to see how this combines with others' experience of automated
> bisections in various environments.  Hope this helps!

Yeah, likewise.

--sRUg/N4PsNSAFS4u
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAmmyB5oACgkQJNaLcl1U
h9BO6wf8DIjFM2QWgE9IXkquY5//VMfsTZq+A4L/6bazmIt8toOatT8lBSGK60QX
kZPFWNVM8OtLfWLC5YOTbVT+sgSQhO3Sn5HF+Lxb5Y0Jz9vr4ucar8sOTKmiOBcs
qwmjpan5KeJKMXCU/q6myVDnMnL3a7r7TroLXLAp5ijMVToEgA1uFyRZ0yRKyTQD
fPOl4LgjA7phoeUe3EMlFm0pyFWJ1s3Rml2nPdyHB1yhXGLfkZHhBExCDEkhC1T0
9+0FASYFRwfclAd5N2WCuore9hNE4W2u5IiJsgxrBPAqZMk7ei8MCrr+Zy8uv6kN
3tvgRwN1hIzk5oitDS987RMQCOHrhw==
=Xxv5
-----END PGP SIGNATURE-----

--sRUg/N4PsNSAFS4u--