Re: Common DBI Driver Test Suite - Requirements

Am 25.09.2013 um 19:27 schrieb Tim Bunce <[email protected]>:

> On Wed, Sep 25, 2013 at 07:02:07PM +0200, H.Merijn Brand wrote:
>> On Wed, 25 Sep 2013 17:28:04 +0100, Tim Bunce <[email protected]>
>> wrote:
>> 
>>>    G9. end users can find the level of DBI API compliance of a driver
>>>        i.e., by looking at the test configuration for the driver
>>>        to see what areas of functionality are configured to be skipped.
>> 
>> I have a hard time parsing that.
> 
> Drivers have various limitations, things they don't, or can't, support.
> It would be unreasonable for them to fail DBIT for those reasons.
> So there needs to be a way for those drivers to tell DBIT that
> certain tests should be skipped.

Riba put that into a

R4: It should be possible to configure which test(case) should be
    run and which not. This should be possible in a manner which
    avoids asking cpantesters 100 y/n questions ^^

Maybe this belongs to that point (Riba should be able to explain better).

For me, a G10 is interesting (handling SQL::Statement as a DBD)

G10: Provide tools/infrastructure to combines several test cases in
     particular order into tests to allow workflows being tested
     and stress tests by looping can be performed.

> And not just drivers. Things like proxies will need to influence
> what's expected to work. This is a key area.
> 
> What G9 is trying to express is that that information (which tests to
> skip) is useful for end users, and probably applications as well.
> So we should aim to design the system in a way that make that
> information available in a useful way.
> 
> Some of that might end up being handled by $dbh->get_info() but there's
> bound to be some that isn't.
> 
> 
>>>    S4. DBIT is not meant to test the underlying database.
>>>        If a driver implements a database itself then it'll
>>>        need it's own tests to provide good coverage of that.
>> 
>> DBIT will offer the interface to test the underlying database. What is
>> important here, as DBIT will offer the possibility to test under
>> different circumstances (like with SQL::Nano or under DBI::Proxy)
>> without having to rewrite all tests over and over again.
>> These test will reside in the test suite of the DBD, not in the test
>> suite of DBIT
> 
> Yes. Or, for drivers that want to share tests, those tests could live in
> modules installed by a distro that the driver declares it's dependant on.

I would prefer the "distributed separately" way. It doesn't have to be
dozen of independent distributions, there could be a few thematically
grouped ones. Like some for ANSI SQL, some for proxy specific ones
(like spreading some requests to see how the scaling is?), …)

Cheers
-- 
Jens Rehsack
pkgsrc, Perl5
[email protected]