Re: features/tokens
Ger Hobbelt <[email protected]>
| Newsgroups | gmane.mail.spam.crm114 |
|---|---|
| Message-ID | <[email protected]> |
The latest experimental GerH build(s) include one(1) 'instrumented'
classifier, which can 'log' words with the feature hashes, but only
directly for so-called monograms (i.e. naive bayes without markovian
in effect. That's a VT matrix defined, for example, as: /vector: 1 1 1
1/ ); it is 'doable' in a technical sense for markovian matrices such
as the sparse bigram ones used throughout crm114 by keeping a 'sliding
dictionary' of logged words. Bla bla bla.
Anyway, this has been done for some VERY SPECIFIC academic research,
which regarded *tiny* input vocabularies (that means: _not_ human
language, like you find in email).
If you need this sort of thing - or think you do - note that this is
technically doable (I've done it); the hard bit is not so much the
'logging', nor the reverse-derivation of words from such tracelogs,
even when inputs are mixed into bigram/multigram feature hashes, but
processing the result, e.g. combined with CSS snapshots for grabbing
feature weights, is on the exponential curve for effort, dependent on
your input vocabulary. Human language has way too many words, even
when you consider the rather limited set used in active speech /
writing by people in general (IIRC ~1000..2000 below academic level).
Cave canem.
>
> is there any way to see the features and/or tokens that are stored in the
> .cfc-files?
--
Met vriendelijke groeten / Best regards,
Ger Hobbelt
--------------------------------------------------
web: http://www.hobbelt.com/
http://www.hebbut.net/
mail: [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H