Re: GerH source distro at hebbut.net: HOW TO compile/compare/merge for Bill et al

On Tue, Apr 1, 2008 at 7:24 PM, Bill Y <[email protected]> wrote:
>  However- how can we test the system if big chunks of it are optional,
>  unless the test system itself gets dynamically reconfigured to only
>  test things that ought to be there?
>
>  This sort of complication is why conditionalizing builds is not something
>  I enjoy.
[...]
>  I tried that once.  What I ended up with was lots and lots of little
>  files that were very hard to maintain.  In short, the testability
>  got worse and maintainability got MUCH WORSE.


Simple: the way I did it, it'll produce a [non]fatal (that's still up
for the board to decide) error message instead for each removed
classifier.

All we need is one run of _Everything_, be it megatest.sh or the
equivalent in a 'make check'/'make megatest'-based set of tests and
the complete output dumped, as before, to knowngood.log

The ones that compile without, say, SKS, get to see the diff for that
one when they check against knowngood: one error message for their
build, some nice "This is SKS" and a set of pR dumps on your
knowngood. I've done it several times like that already. Child's play.

I prefer to split up megatest.sh and incorporate it into the 'make'
system under 'make check' and/or 'make megatest', so the tests can, as
one desires, be run separately too.
Meanwhile, using that approach, there's no maintainance hazard with
'lots of little files': I have a set of .input files replacing those
EOF<<< shell redirects because I need hem for Win32 *anyhow*, but
those files don't require maintainance at all as these are FIXED input
references. You haven't changed any of those lines in megatest
anywhere in the last couple of years, so I can safely say they're
stable as granite rock and I expect them to remain so.
New tests may use the same input files, though I'd prefer each test to
have their own input file, just so one can 'mess' with them when
testing for something specific.
Documentation should not be used as test input reference material as
documentation will change over time - hopefully :-) - , while test
results should be as stable as humanly possible. Hence also my move to
feed the classifiers a '.input' copy of the current files: I've found
that off-by-one and other 'minor differences' in output don't improve
confidence in the results out here, not to put too fine a point on it.
And some of those 'tolerable differences' have unearthed code bugs
that disturbed me.


>  I am also unconvinced that there is very much savings in the idea of
>  leaving out some classifiers.  The binaries just aren't that big
>  (about 32Kbytes each) and unless you're building a VERY small embedded
>  system, you're just buying yourself a world of untestable hurt by
>  taking things out.

I've found that some folks don't like to have 'experimental' stuff
lurking in their production software. I can argue for it on technical
grounds for years on end; the bottom line is that it's an emotional
choice. It's got 'experimental' in it? They don't like it. They get
the GUARANTEE there's only 'stable, proven technology' in there?
They'll breathe again and be happy. They'll love 'ya.

The CRM_WITHOUT_<CLASSIFIER> #defines give everyone that choice to
feel good about it. Personally, I'd rather run it with everything on
there and I don't mind a dead weight classifier, but it's not the way
it works out there in the lands that I have visited.


I hope I'm not bothering you with all this.



-- 
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web: http://www.hobbelt.com/
 http://www.hebbut.net/
mail: [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace