Re: On running test suites during build - and bootstrapping issue

Le mer. 27 mai 2026 à 22:18, Simon Josefsson <[email protected]> a écrit :

> Jérémy Lal <[email protected]> writes:
>
> > Hi,
> >
> > running a test suite during build is something that feels more and more
> > useless, given debci.
>
> While I think building with 'nocheck' on buildd's at some point in time
> would be reasonable, I still (only occassionally) see different kind of
> failures during buildd and debci/autopkgtest from self checks.
>
> So running checks during build-time do catch some errors that wouldn't
> be caught by debci/autopkgtest alone.
>
> As a first step towards the goal above, would it make sense to make
> testing migration block on a 'nocheck' build producing an identical
> *.deb?
>
> Maybe the reproduce.debian.net builds could be run with 'nocheck' which
> would effectively achieve that goal.
>
> I believe this would catch packages that produce different *.deb's
> depending on nocheck setting, which I believe still happens (I've both
> introduced and fixed a couple of such mistakes).


Interesting, because I started the day wondering nodejs build failed
on tests.reproducible-builds.org (a test failure during build).

Combined with the input from Paul, I see that tests need to be split in two
sets,
if not already:
- tests that can be run without problematic build-dependencies, or without
problematic assumptions about the environment. We want those during build,
most of the time.
- other tests that can run well in debci



> /Simon
>
> > It also adds a bootstrapping problem when (say, nodejs, surprise) the
> > package build-deps pull
> > a test dependency that has the package as a dependency.
> > I'm having that issue right now on ppc64:
> > https://buildd.debian.org/status/package.php?p=nodejs
> >
> > I already asked about this, but I think it was before we had migration
> > rules dependent on debci results.
> >
> > Is it time to let the tests be done by debci only ?
> >
> > An alternative question is, is there a way to automate the bootstrapping
> > process ?
> > We have build profiles, but no way of processing them automatically, do
> we ?
> >
> >
> > Jérémy
>