Re: size of variables in hyperspace?

> Ouch... in fact, I'd be worried that the boilerplate around it (especially
> the javascript) may be including "telltales" that are dominating
> the decision process.

does that mean hyperspace does not limit itself to storing the unique
features of each category, like osbf does?

usually, in machine learning, it is a good idea to include as much
information as possible, including things that would only confuse a
human being...



> I think you might want to strip that stuff out.  Anything that
> a human looking at the screen wouldn't see as text is a candidate
> for removal.
>
>   using several different statistics files for each category does not
>   sound like a good idea. i already have 9 categories (plus 'uncertain'
>   and 'out', which are fail-categories), and i think the high number of
>   categories makes it harder for the classifier. we have ruthlessly
>   reduced the number of categories from 29 to 9.
>
> Maybe things would work better if you got rid of the boilerplate.
>
> It's a worthwhile experiment.
>
>   - Bill
>
>   On Tue, Jun 9, 2009 at 8:07 PM, Bill Yerazunis <[email protected]> wrote:
>   >
>   >   From: Thomas Michael Hagen <[email protected]>
>   >
>   >   thanks!
>   >
>   >   making the thickness thinner should help.
>   >
>   >   you're saying splitting the .chs-file is a good idea, but can i merge
>   >   them back into each other again afterwards? what tool do i use?
>   >
>   > No need to - just use all of the spoN.chs files together on the
>   > CLASSIFY line.
>   >
>   > Merging them together is itself problematic, because again you'd
>   > have a file bigger than 2^32 bytes.  Although it's possible to
>   > have files larger than that, right now CRM114 doesn't jump
>   > through the necessary hoops to do so.
>   >
>   > See:
>   >
>   >  http://en.wikipedia.org/wiki/Large_file_support
>   >
>   > for a blurb on it.
>   >
>   > To be honest, I find it astounding that you've got training
>   > data of more than a few tens of megabytes... which is why I
>   > am a bit aback about thinking about data of plural gigabytes.
>   >
>   > If I knew up front you'd be storing gigabytes, I probably would
>   > have designed the hyperspace data structures differently.
>   >
>   >     - Bill Yerazunis
>   >
>
>

------------------------------------------------------------------------------
Crystal Reports - New Free Runtime and 30 Day Trial
Check out the new simplified licensing option that enables unlimited
royalty-free distribution of the report engine for externally facing 
server and web deployment.
http://p.sf.net/sfu/businessobjects