Re: Enabling additional KUnit tests in KernelCI?

On Thu, Mar 12, 2026 at 11:34:51PM +0000, Mark Brown wrote:
> On Thu, Mar 12, 2026 at 02:51:45PM -0700, Eric Biggers wrote:
> > On Thu, Mar 12, 2026 at 08:21:26PM +0000, Mark Brown wrote:
> 
> > > We should be running KUnit tests:
> 
> > >   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/jobs.yaml#L2153
> 
> > >   https://github.com/kernelci/kernelci-pipeline/blob/a3b622cfee641e26bed1906b29ed7065bee45921/config/scheduler.yaml#L1750
> 
> > Which ones?  Does anything need to be done to add new tests to the list?
> 
> > Note that even if "all" tests are being run (via
> > CONFIG_KUNIT_ALL_TESTS=y), that includes only the tests whose
> > dependencies are met.  There can be additional kconfig dependencies.
> 
> Whatever is enabled by the KUnit runner tools/testing/kunit/kunit.py
> (which makes x86_64 a bit of an unfortunate choice given how slim theilr
> defconfig is).  The theory is that the KUnit configs will do the right
> thing, it looks like we're not currently pasing --alltests though which
> we should - I'll look at a patch for that tomorrow.  If you need arch
> specific options that aren't enabled by the platform's defconfig that
> gets a bit awkward.

Yes, the tests run by kunit.py depend on the args passed to it.  It
looks like without any arguments it runs all tests available with
defconfig + tools/testing/kunit/configs/default.config.  With
--alltests, it runs all tests available with defconfig +
tools/testing/kunit/configs/all_tests.config.  Both set
"CONFIG_KUNIT_ALL_TESTS=y", so the difference is just what additional
options get enabled to fulfill the test dependencies.  all_tests.config
enables more dependencies than default.config.

So I think KernelCI should start passing --alltests to kunit.py, and I
should get the dependencies for all the crypto and CRC tests added to
tools/testing/kunit/configs/all_tests.config.  Then KernelCI would pick
them up automatically.

Same for any other kernel subsystem: it seems maintainers should get
their test dependencies added to all_tests.config.  (Unless those
options are arch-specific, as you mentioned.  The crypto and CRC options
are all generic though; just the implementations differ across arches.)

> > > though like you say something seems to be going AWOL with at least the
> > > reporting, I can't see any results either.  The job that's configured
> > > there is to run on x86_64 rather than UML so is probably what you're
> > > looking for in terms of the tests?
> 
> > The kernel has optimized crypto and/or CRC code for the following
> > architectures: arm, arm64, mips, powerpc, riscv, s390, sparc, x86.  In
> > many cases there's also a finer division based on CPU features.
> 
> > So I'm looking for testing on as many platforms as possible.  But some
> > x86_64 platform is a good start, certainly better than nothing.
> 
> The KUnit runner uses qemu as standard which I guess is probably fine
> for an initial pass, we would need to do a bunch more plumbing to run on
> actual hardware and pull the results out.  We'd also be restricted by
> what's available to us, at the minute we've got arm, arm64, riscv and
> x86 systems with a bit of an embedded focus.

It seems kunit.py uses UML by default, though it accepts an --arch
argument which enables QEMU.  So something to think about for KernelCI
would be to run all KUnit tests with different combinations of --arch.

Of course, running the tests natively on different hardware would be
even better, when such hardware is available.

- Eric