Re: Mixed 64-bit system GerH binaries / BillYscripts --> two-sided training? YES!
Bill Yerazunis <[email protected]>
| Newsgroups | gmane.mail.spam.crm114 |
|---|---|
| Message-ID | <[email protected]> |
What Fidelis said is indeed what I was going to suggest if it were
not for a bad net connection and a very unhappy 11-month-old.
But yes, try OSBF for now.
Actually, one "good" justification for going to a "train both
positive and negative examples" in libcrm114 is that _in the special
case of just two classes_ . If you know (ahead of time) that you
only want a spam filter (and no advanced email sorting) then
you train one statistics file for spam-matching (positive training)
and that's all.
Then, classifying with that file will yield a
"Best match to file #0 (spam.css)"
if it's spam, and the (new)
"Best match to file #-1 (none_of_the_above)"
if it's not spam.
Advantages:
1) most users will need only 1 statistics file (not two)
2) if we go with 32-bit features, then the size of the statistics
file for Markov / OSB / OSBF will stay the same, but since you
only need one, then the overall disk footprint and cache requirement
will drop by a factor of two.
3) you can still add other file types later. It just may take a
little while to train them in.
4) there will no longer be any difference in how you train classifiers;
Markovians will train the same as SVMs and neural nets. No more code
dependencies!
5) - and you will be able to mix different classifiers in the same
CLASSIFY statement.
So, Ger, what does this look like in a portable 32/64-bit C:
typedef struct mythical_new_bucket
{ gimme_32_bits hash;
gimme_32_bits in_count;
gimme_32_bits out_count:
} standard_bucketp;
Do I have to use uint32_t (which can't be printed by printf
without casting? Or is there some other incantation that Just Works.
- Bill Yerazuins
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/