Re: GerH source distro at hebbut.net: HOW TO compile/compare/merge for Bill et al

   From: Paolo <[email protected]>

   On Fri, Mar 28, 2008 at 10:16:33PM +0100, Paolo wrote:
   > 
   > Now with .txt, test takes 16m44s and seems to work, except for diffs in 
			 ^^^^^^^^^^^^
   Note:

   $ make megatest

   ...
      to exercise much of CRM114 under TRE.  This takes about
      1 minute to run on a 1.6 GHz Pentium-M laptop.  Please
      ^^^^^^^^
   ...

With the new tests, it's closer to three minutes, and I'm working
on cutting that down.

However- how can we test the system if big chunks of it are optional,
unless the test system itself gets dynamically reconfigured to only
test things that ought to be there?

This sort of complication is why conditionalizing builds is not something
I enjoy.

I am also unconvinced that there is very much savings in the idea of 
leaving out some classifiers.  The binaries just aren't that big
(about 32Kbytes each) and unless you're building a VERY small embedded
system, you're just buying yourself a world of untestable hurt by
taking things out.


   better correct the expectation here, if it's going to exercise all the 
   classifiers, we know it'll take a lot more than just 1'.

   BTW, that should probably make to depend on actual classifiers included.
    And megatest.sh should skip those excluded.
    Better yet, megatest.sh should be splitted in eg. 
    - regex_test.sh
    - strnops_test.sh
    - class-<name>_test.sh
    - ...

    then known / test logs would be combined for the grand diff.
    Finally, it'd help somewhat to have timing in the known / test
    logs as well, since speed isn't a marginal performance index here.
    A known/stable classifier timing would make for the ref
    value/scale.

I tried that once.  What I ended up with was lots and lots of little 
files that were very hard to maintain.  In short, the testability
got worse and maintainability got MUCH WORSE.

    - Bill Yerazunis

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace