Re: Rethinking how ext/ is tested

2009/2/27 Michael G Schwern <[email protected]>:
> demerphq wrote:
>> Id prefer we find a different solution. Im not sure what it is, but
>> having the tests all run in one block is a very desirable feature.
>> What you are saying is that to know what is broken we have to review
>> many test summaries, this sound sensible if you are thinking about
>> tests only being broken in your module, but its not sensible when a
>> core change has broken a nice chunk of the modules out there. You want
>> to be able to look at one summary at the bottom of all the failures.
>> Reviewing a stack of summaries with different counts and etc doesnt
>> sound fun.
>
> Eh, we already have to search around in that huge list of test results anyway
> to get results.  It's not like this is going to be much worse.  We already
> deal with a similar issue when building the ext modules.  In the case of ext's
> tests, they're all going to be outputting in the same format to it's not like
> we're introducing some wild west of tests here.

No we dont, we just look at the summary line which provides all the
information we need nicely, er, summarised.

> But, like I said, the formatting can probably be improved.  We've got a known
> version of Test::Harness and can play with HARNESS_OPTIONS to transparently
> tinker with TAP::Harness.
>
> If we use the traditional make subtest setup, testing will stop once the first
> ext module fails.  While this would certainly make them easier to find, this
> is probably not desirable given the Perl tradition of always running all
> tests.  A small wrapper would need to be put in place to ensure all the tests run.

It is definitely not desirable.

> The important thing is the core change from running the ext/ tests manually to
> using each module's own "make test".  This will make integrating and
> developing dual life modules far easier by eliminating a key difference in how
> ext/ and CPAN modules are developed.

Personally im not fussed, and consider the damage to the error
reporting outweighs the minor convenience of dual lifed modules
avoiding some boilerplate.

Yves



-- 
perl -Mre=debug -e "/just|another|perl|hacker/"