Re: Bug#1024971: pybuild: should fail when the result of running tests is "Ran 0 tests in 0.000s"

Hi Julian (2024.09.09_11:19:51_-0400)

a year later...

>On Mon, Sep 09, 2024 at 08:31:52AM +0000, Stefano Rivera wrote:
>> > Hi Julian (2024.09.08_21:33:49_+0000)
>> > > I built 6440 packages build-depending on dh-python in one way or
>> > > another. 1483 failed, and 1124 of them say "NO TESTS RAN" in the logs.
>> >
>> > My guess is that most of these 1124 have no tests at all, rather than
>> > having a misconfigured setup.  A unittest is the pybuild default test
>> > framework, unittest is used and fails to find any tests, hence all of
>> > these failures.
>>
>> Yes, almost certainly.
>> [...]
>>
>> I would leave unittest as the default runner, but without missing test
>> detection.
>>
>> That's a slightly unexpected behaviour, but it makes the default case
>> work.
>>
>> Downside is that you have to opt-in to missing test detection. Maybe we
>> can have a lintian tag for that?
>
>That seems a bit heavy to ask for.
>
>Is there any way of identifying those packages that do genuinely use
>unittest?  If there are not that many of them, then implementing a
>--test-unittest option would be a good way to go.  I would imagine the
>following timeline:
>
>(1) --test-unittest is introduced as an option to explicitly select
>unittest as the test framework.  When --test-unittest is specified,
>the test will fail if no tests are found.  unittest is still used as a
>fallback test framework; in this case, the dh_auto_test call will
>succeed if no tests are run.

This feature has now been added in dh-python 6.20251221.

So, if anyone really wants to use the unittest runner, please declare 
--test-unittest (or export PYBUILD_TEST_UNITTEST=1).

Stefano