Re: CRM-114 customization for chocolate?

Tracy Reed <[email protected]> writes:


> On Fri, Oct 14, 2011 at 08:35:54AM -0400, [email protected] spake thusly:
>> Multi-class differentiation is a _much_ harder problem. =20
>
> I think I brought this subject up on the list once, a few years ago. And
> I'll re-raise the same question I did then which I seem to recall we
> agreed would work in principle but the accuracy was never mentioned so
> let me ask again but frame it differently:
>
> Would it be possible to get better than 90% to take a kind of
> binary-tree sorting approach and cut the corpus in half each time? The
> downside is that a human would have to intelligently arrange the order
> in which it was subdivided which may defeat the purpose.


And with a few years of actual expereimentation: "No, it doesn't help."

We tried it.  You actually get better accuracy doing 20 repeats of
one-class-versus-all-others than to do the five-level logarithmic sorting
tree.

And it's not even a publishable result.  :(

> For example, I would separate my computer mail from my non-computer mail
> at a first pass. Then I would sort computer mail in terms of
> Linux-related stuff vs programming stuff etc.  I might sort non-computer
> mail in terms of flying (one of my other avocations) vs other stuff etc.
> Dividing it up based on which sorts of things tend to have the most
> distinct vocabularies/topics.

You'd think so - given that most of the classifiers are linear
classifiers and "should" have more trouble splitting (say) linux
compiler stuff from "general computing" stuff if there were (or 
weren't) other stuff in the mix too.

But reality and experimentation showed no such luck.  The reason
is probably the same reason that linear and nonlinear text 
classifiers are about equally matched in accuracy.

Nobody _really_ knows that, but I'll wager that human natural
language is so high-dimensional that there are a huge number
of linear partitionings that all work just fine.

Note that figuring out a better partitioning algorithm IS
a publishable result.  :)

   - Bill Yerazunis


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct