Re: Multiple categories and more

AOn Fri, Aug 22, 2008 at 03:41:03PM -0700, James Lee wrote:
>    classifiers.  OSB was trained using SSTTT, OSBF using DSTTTR and
>    HyperSpace using TOE.  Preliminary results showed that OSB=20=

you can use such tecniques with all those classifiers.

> <osb unique
>    microgroom> was most accurate.  Here are my questions:

good to know :)

>    - As I increase the number of categories, will classification process
>    slow down?  What about accuracy?  Also, theoretically speaking, is

sure - well, depends: if you have n classes, then add +1 same size, you add
~+1/n of exec time; if you scale down class size growing n, or do some
magic decision tree through the classes, things get more complicated.

>    there a limit on how many categories I can classify into?

iirc compile-time limit was set at 128 classes; but you can change it, then
limit is set by avail resources.

>    - If the number of categories becomes an issue, will be it better
>    (both performance and accuracy wise) for me to classify into two
>    categories at a time and pick the highest pR?  For example, classify

who knows? depends on problem at hand, data, training method ... given a 
problem, devise a few learn+classify schemes and try - and report back
the results ;)

>    - The pR score for HyperSpace seems to be lower than OSB and OSBF, is
>    this normal?

yes, all classifiers have there scale and shape wrt in/out relation, as well
as optimal thresholds (usually to be tuned in 'production').

>    - Is there a such thing as optimal statistics file size for each
>    classifier?  It appears that the default size for OSB is 6M, OSBF is

depends on data; usually you can set a class size where purging of aged 
and/or marginal entries ('microgrooming') is minimal, < 2..3%; there are
theories about such 'optimal brain damage', which usually imply a careful 
choice of what to discard an when; at present the cssfile formats doesn't
store explicitly such infos, only some guess/heuristic is applied to drop
oldes and least significant entries. 

>    - Is it possible to "over-train" the classifier?  As I train more and
>    more, since the file size is limited, does it simply throw away the
>    oldest data?  Or is it more complicated than that?

see above - more complicated, and more simplistic at same time: we hopely 
throw away the 'right' stuff, but on a heavily 'groomed' class who knows 
what happens. What has been observed, is that - yes - over-training might
happen - which leads to over-fitting - and sensitivity depends on 
classifier, data, and datawindow size, eg (for email) HS needs smaller 
chunks than eg OSB (say 4k vs 15k), else it'd over-fit quite easily.
Likely that's related to HS dimensionality.


HTH

-- 
paolo


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/