Re: Release process (was Re: DarwinPorts 1.3.1)

On Aug 1, 2006, at 9:55 PM, Paul Guyot wrote:

> The recent fiascos of releases 1.3 and 1.3.1 show indeed that there  
> is something wrong with the release process.
> But it is not (only) a matter of the timing of release. It's mostly  
> a matter of testing. [ ... ]

And here, I agree with you 100%.  Where I think you go off in the  
wrong direction is with the rest of the points you make.

I'm going to sound like a broken record here, but the real problem is  
that the DarwinPorts project simply doesn't eat its own dogfood.  It  
relies on the user community to do that for it, and no number of  
release candidates or changes to how the bits are actually released  
(CVS, etc) will change that one bit.  Yes, developers make terrible  
testers, but for a collection of 3400+ ports coupled with 40,000 lines  
of Tcl and C code supporting it, users make even worse testers because  
there's no way they can consistently cover all of that ground even  
once, much less on a regular basis, and the end result is a total hit- 
and-miss journey through the collection as users stumble across broken  
ports more or less at random (or test the really popular ports and  
ignore the rest).

DarwinPorts will never be more than an intellectual curiosity until it  
evolves the infrastructure for clean-room building each and every port  
on a regular basis, with known combinations of "base" bits, and  
resolving (or at least reporting) all of the errors encountered.    
DarwinPorts is not a paint program or a word processor, and you can't  
get comprehensive testing of it just by releasing it to the user  
community and letting them bang on it for a while.  It's something  
which requires an entirely different level of testing (and has never  
had).

- Jordan