Re: Fwd: Re: ifile vs. Graham modified naive bayes?

> [email protected]:

> I did _not_ propose CDB for "faster update;" only for "faster access."

oh, i think i understand!  you're the "ifile -q" type of user, right?  and
you prime the database from carefully selected messages?  well, i was
thinking in my own terms:  "ifile -Q" and refiling messages later that were
classified wrong.  for "-q", the CDB approach is good.

> Here's a perfectly good ".idata compiler.":
> 
> #!/usr/bin/perl
> use CDB_File;
> my $idata = new CDB_File ("/home/cbbrowne/.idata.cdb", "cdb.tmp");

i also didn't know that there's a perl module for CDB databases.  your
proposal sounds good, except for:

> It "compiles" a plain old standard .idata file 8MB in size into a
> very-slightly-larger .cdb file in about five seconds.

how on earth do you get a dataabse this large?  how many categories do you
have, and which options do you use?

> If you look, you'll see that "freecdb" is included in Debian, as are
> python-cdb, libcdb-file-perl, libvpopmail-freecdb, dbskkd-cdb, and
> skkdic-cdb.

that i also was not aware of.  mr. browne, you won the prize! :)

> OK, how about I make it more nearly literal.  Add to some function like
> ifile_lexer_simple_postprocess_word(), the code:
> 
> int wordlen = strlen(buf);
> if ((buf[0] == 'M') && (wordlen == 61)) {
>     buf = "lookslikeuuencoding";
> }
> if (wordlen == 76) {
>    buf = "maybebase64";
> }
> if (wordlen > 35) {
>    buf = "BigLongString";
> }
> 
> [...]
> 
> lookslikeuuencoding 475 23:1791 24:2477 27:82
> maybebase64 470 7:2384 8:921 9:751
> BigLongString 74 1:7 2:35 7:17 9:47 23:57 24:171 27:17

very good!  this is really making perfect sense!

clemens