Re: Some questions setting up latest CRM114

   From: Leonard Lin <[email protected]>


   > I'm finding that a pR threshold of +/- 5.0 is working better these
   > days, as it allows micro spams to be caught very easily (microspams
   > are spams with just a line of text and single URL as the payload)

   Thanks for the tip I'll take a look at the thresholds and see what .   
   Do you know offhand about whether the X-CRM114 headers are all ignored  
   when processing in mailreaver?  I'm looking at the reaver/lib sources  
   and see some mangle calls that I think might be doing the right thing  
   but I'm not sure.

I took a look as well- they appear to be in the outgoing
/updated/confirmation path, not in the "mung before learning" path.

So, the answer is "no, not ignored!"

   Also, and I guess I'll ask it here instead of crm114-discuss since  
   there doesn't seem to be a lot of active responses there, but I'm  
   interested in also using crm114 for classifying other datasets.

   * Has anyone used crm114 to successfully w larger (say, 1-10M item)  
   datasets?  Does it blow up either on insert (is it linear speed or  
   does it slow down?) or for classifying?

~100K has been done at TREC.

The datasets grow only when you have to train, so if you use TOE
or SSTTT, the growth is logarithmic at worst and asymptotically 
limitend for most data.

   * And secondly, the hyperspace classifier seems really interesting  
   because it sounds like (in theory) it would be good for different  
   types of classification - I've seen some search results for n-way  
   classification -- are there docs/examples for that?

Yes, it does.  

Take a look at "gatlingsort" for an N-way classification.

   * Also, are there semantics for pulling "relatedness" (i.e. semantics  
   for finding distance, or retrieving nearest neighbors)?

At this point, no.  The distance is "hidden" from the user, and the
neighbors are stored only in a hashed form that (to an extent 
intentionally) obfuscates the original text.

However, CLUMP will be able to do that for you.  If you want to 
get into coding C, let me know and I'll give you the design
briefing.  

	 - Bill Yerazunis

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace