Re: Mixed 64-bit system GerH binaries / BillYscripts

The whole 32/64 bit thing is tipping me in favor of a significant
restructuring of the classifier code.

The reason for the initial size of the .css files was that long longs
(that is, a 64-bit int on a 32-bit machine) were not behaving well 
way back when  (where "way back when" was in the last millenium).  So,
.css files used a pair of 32-bit numbers to simulate a 64-bit hash.
That gained us about 5% accuracy improvement at the cost of doubling
the size of a hash table.

But then OSB came up (thanks Christian!) and that cut the size of
the hash tables by a factor of four.

Now a whole new millenium (and almost a whole new decade too) and
now there's no need to use a pair of 32 bit ints any more,
so I have been kicking around a restructuring.  Among other things,
it would make libcrm114 easy to create (in fact, that's where the 
/usr/bin/crm binary would get the classifiers)  

Now, here's the downside: to get a good library interface, it might
be necessary to make _all_ classifiers require two-sided training.

That is, you would train "affirmatively" to every correct class, and
"negatively" to every incorrect class.  This doesn't seem horrible,
and it does also give us a multivalent classifier - that is, you 
could train your good email with the OSB Markov, your spam with
Winnow, and your binary por... er, fine art photos with bit entropy,
and all of these classifiers could be used together in the +same+
CLASSIFY statement.  

Between that and an SVM rewrite, that's what *I* have been up to.

I'll post a high-level design and see what you folks think of it.
I've only been doing design (i.e. pseudocode) so there's no big
investment of C coding in this, and if it's the wrong thing, now
is an excellent time to speak.

I won't start actual coding until I hear what people have to say
(and I hope Ger has a chance to chime in on what will make portability
an eas(ier) thing with respect to X32/64

     - Bill Yerazunis

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/