Re: crm114 on 64 bit SuSE

On Tue, Aug 12, 2008 at 10:07 AM, Thomas Spahni <[email protected]> wrote:
>> Don't know where yet, but definitely a screwup in my mail*.crm scripts.
>
> Yes, when find is syscalled it gets a -name argument with / at the end.

What I need to (re-)discover is why I did that. (I seem to recall an
issue I had with double / slashes in some paths in there a while back,
but I didn't write it down and other stuff has pushed it out of my
brain FIFO, so I'll have to pay for that.


>> make megatest_ng
>
> I could not find a target 'megatest_ng' in the Makefile
> (crm-114-20080502-BlameJason-Ger-2423) but there is /tests/megatest_ng.sh.
> How do I run that?

Ah, yeah, quick check revealed that. Erm, answer is 'no' on that one
(at least: I am not sure megatest_ng will work from the tests
directory in that release already), so here's the way that _should_
work, because that's where it all started (as I moved from Bill's
single source dir to moving tests to a separate tests/ dir later):

cd src/
make megatest_ng

and you should have the ./src/megatest_ng.sh file in the extracted
contents of the GerH distro /before/ you run the above two lines.


> seems to run ok. Some diffs for pid numbers as expected.

yup.

FYI: notice a few things in that torrent of OK's and other lines:


(1)

everywhere where you see the words
  AWK filter tolerances
it means the test&compare scripts have been passed some (maybe none,
maybe many) 'tolerances': each of these is a VERY crude 'allowed
range' for a particular item reported by CRM114 during that particular
test; any results within that 'tolerance' are thought to be 'OK';
though some are technically / theoretically disputable.

For example:
  AWK filter tolerances / args: -v entropy=100 -v jumps=10 -v prob=1.0 -v pR=4.0
says this:
when CRM114 reports an 'entropy' number within +/- 100 of the
reference value, it's OK.
'jump' numbers may be +/- 10 apart.
'prob' probability numbers may differ +/- 1.0 and still they're fine, while
'pR' values may be as much as +/- 4.0 apart.

(Due to the extreme crude way the scripts check this, this is not
entirely true: where I said above " any results within that
'tolerance' are thought to be 'OK' " it should read as "  MANY results
within that 'tolerance' are thought to be 'OK' ". But that's some
rather technical AWK-script related issue, which I hope to fix at a
later date. Currently, the tolerance settings are wider than they
should/could be, due to this test script issue.)



(2)

Another item to note are those lines saying:

  FAIL(OVERRIDEN:OK)

which are failed tests, but are considered 'known issues' by me (which
leads to the conclusion that 'Bit Entropy' and 'CLUMP' are definitely
off limits for use) and marked as such in the tests/Makefile.am



(3)

Also note that 'Neural Net' is NOT tested: the tests take a bloody
long time to complete and there are still issues in there that make it
a very risky one for regular use anyhow. (I consider a classifier
'stable' if I get identical (or at least extremely close) numbers on
/all/ my platforms, 32- or 64-bit. Few do.)



>> > Beware of not mixing use of those 'branches' with same cssfiles (unless
>> > you know what you're doing, that is).
>>
>> Second that!!
>
> I compiled in different directories an run megatest right after compiling.
> This should not mix css files, right??? And it's using the new copy of
> crm114 and not the old one hanging around in /usr/bin ???

GerH ./configure patches the CRM scripts 'bang line' to ensure the
currently built binary is tested.

As Paolo said: looks fine, because you get the expected copyright
statements in both vanilla and GerH circumstances, so I don't worry
about that.


> Any ideas why count of messages learned goes up by 2 after learning one
> message? I never observed this on 32bit systems.

Hm, that's vanilla/BillY on 32-bit and what classifier?
I wonder what GerH does -- never saw that happen here but then (a) I
never thought to check for that bit in particular and (b) I don't use
the mail* scripts, so you /may/ see an artifact of those: depending on
your mailfilter.cf settings, mailtrainer.crm /can/ train a message
multiple times -- and each time a message is trained it is counted +1,
so two rounds of training would show up as '2 documents trained' (even
while it's the same doc twice: CRM114 itself doesn't know that.
But that's me guessing from across the field...



On a closing note: if you want to play with the GerH binaries (and
don't be bothered with those mail*.crm script errors of mine), you
should be able to use the GerH binary as a 'drop in', i.e. using
Bill's original crm scripts and a GerH binary. Like Paolo warned
about: when you do this, you MUST 'rm -f *.css' your CSS files as GerH
and vanilla/BillY binaries produce different format CSS files on
64-bit (I _should_ be able to use Bill's CSS files produced on 32-bit
boxes, but not the other way around: GerH CSS files can never be used
by BillY binaries, due to an extra versioning/platform-identification
header block I added). In short: want to try? delete all CSS files.
replace binary with a GerH one and it should work. If not, please
notify me.

Extra note: older vanilla scripts will report a syntax error when run
with GerH binaries; AFAIK this has been fixed completely in the latest
vanilla/BillY scripts.




-- 
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web: http://www.hobbelt.com/
 http://www.hebbut.net/
mail: [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/