Some questions

Hello all

I don't know if the list is the right place to ask, but I try. I would like to understand better how SBPH and OBF is working. According to the CRM114 documentation I have made the following table:

W = Weight
C = Complexity or matched words
S = Sparse
F = Features

SBPH:
----+---+---+---------------------------------
  W | C | S | F
----+---+---+---------------------------------
  1 | 1 | 0 | jumped
  4 | 2 | 0 | fox jumped
  4 | 3 | 1 | brown <skip> jumped
 16 | 3 | 0 | brown fox jumped
  4 | 4 | 2 | quick <skip> <skip> jumped
  4 | 4 | 2 | quick <skip> fox jumped
 16 | 4 | 1 | quick brown <skip> jumped
 64 | 4 | 0 | quick brown fox jumped
  4 | 5 | 3 | the <skip> <skip> <skip> jumped
 16 | 5 | 2 | the <skip> <skip> fox jumped
 16 | 5 | 2 | the <skip> brown <skip> jumped
 64 | 5 | 1 | the <skip> brown fox jumped
 16 | 5 | 2 | the quick <skip> <skip> jumped
 64 | 5 | 1 | the quick <skip> fox jumped
 64 | 5 | 1 | the quick brown <skip> jumped
256 | 5 | 0 | the quick brown fox jumped
----+---+---+---------------------------------

For calculating the weight I used the following OpenOffice.org Calc formula:
=2^(2*(Complexity-Sparse-1))

OSB:
-----+---+---+---------------------------------
   W | C | S | F
-----+---+---+---------------------------------  
3125 | 2 | 0 | fox jumped
 256 | 3 | 1 | brown <skip> jumped
  27 | 4 | 2 | quick <skip> <skip> jumped
   4 | 5 | 3 | the <skip> <skip> <skip> jumped
-----+---+---+---------------------------------  

For calculating the weight I used the following OpenOffice.org Calc formula:
=(5-Sparse)^(5-Sparse)
I used 5 as the window size.


Is the above table right? If not, what errors have I made?

If I understand the documentation right, then the weight is used to calculate the probability for a class (either spam or ham). Can one explain me what role the count of a feature inside the CRM Sparse Spectra file plays? I mean assuming the feature "fox jumped" and assuming that feature was found 500 times in the class ham and 100 times in the class spam. How does that 500 and/or 100 count play a role in calculating the probability? Does that even play a role or is it unimportant how many times a features has been learned before?

What happens when CRM114 is saving the data inside the CSS file? Does it save the feature "quick <skip> <skip> jumped" as a concatenated hash value (replacing the <skip> with some kind of delimiter) or does it save just one hash (for the above feature) and at run time after parsing and hashing a message (and therefore knowing the complexity and sparse of a feature) it compares the calculated (in memory) feature with the one found in the CSS file?

I mean:
Assuming the feature: "quick <skip> <skip> jumped"
Assuming hashing "quick" would be: "1111111111111111"
Assuming hashing "jumped" would be: "2222222222222222"
Assuming hashing the whole feature would be: "12345678ABCDEF09"

Does CRM114 now save "12345678ABCDEF09" or does it save "1111111111111111" + some kind of delimiter for <skip> + some kind of delimiter for <skip> + "2222222222222222"?

Another thing I don't understand is this slighting window of 5. Assuming the following content: 1 2 3 4 5 6 7 8 9 0
Does the window now slide like this: "1 2 3 4 5" and then "6 7 8 9 0"
Or does it overlap: "1 2 3 4 5" and then "2 3 4 5 6" and then "3 4 5 6 7 8" and then "4 5 6 7 8 9" and then "5 6 7 8 9 0" and it quits because it can not allocate again 5 words.
Or how is this slighting window working?

Sorry for all the stupid questions, but I really would like to better understand how the things are working inside CRM114. If the list is the wrong place to ask that questions and you know where I could ask them, then just let me know. Or maybe you know some document better describing how things work in CRM114 then just let me know.

// Steve
-- 
Neu: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate + Telefonanschluss für nur 17,95 Euro/mtl.!* http://portal.gmx.net/de/go/dsl02

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & 
iPhoneDevCamp asthey present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://www.creativitycat.com