Re: Why do we restrict our egg testing?

On Sep 26, 2007, at 7:57 PM, Roger Ineichen wrote:

> Hi
>
> Can anybody tell me why we restrict our test setup
> in zope eggs and only use a subset of package for
> our test setup?

It isn't practical, during development, to test all of the eggs that  
might be affected by a change, which is, BTW a *much* larger set than  
the old Zope 3 tree.

For unit testing of the egg under development, there isn't much point  
in running other tests.  For functional testing, you want to test the  
package's layer and, for an application a set of components that may  
be configured quite differently than the Zope tree.

I think there is value in testing some universe in an off-line way  
using something like buildbot,

> Why do we not use a Zope3 meta egg which contains all
> our zope packages as a test base. This whould allow
> us to test the same we have in the zope3 trunk and let
> us run *buildout/test -s zope* from within each egg.

I personally don't see much value in that, especially considering the  
effort involved.

> what is the builbot doing right now? Does the builbot
> still runs test on the trunk? Or does the buildbot test
> the eggs?

Unfortunately, buildbot is a bit of a pita.  :(  It seems to be high  
maintenance.  If someone wants buildbots to run, they should help or  
make it happen.  Christian Theune has a buildbot setup that ran tests  
for all of the projects in the repo.  I think this was *very*  
promising, but I don't know what the current status is.

> Is this a bad idea?

Depends on what "this" is.  I think running all of the tests in the  
old Zope 3 tree whenever we change a component is a waste of time.

If "this" is having an automated system for testing a wide collection  
of packages to check for dependency breakage, then "this" is a great  
idea, but potentially a lot of work.

Jim


--
Jim Fulton
Zope Corporation