Re: Mixed 64-bit system GerH binaries / BillYscripts --> two-sided training? YES!

On Sun, Aug 31, 2008 at 04:46:28PM +0200, Ger Hobbelt wrote:
> dang. 550 This message matches a blacklisted regular expression
> (pharma ban) (state 18). :-( Okay, find&replaced the bloody blue pill.
> Hoped the system was using CRM114 instead of brain-lobotomized

;)

> New code: first check if the hash ('word') exists ON THE OTHER SIDE:
> if it does, DECREMENT (-1) the weight OVERTHERE (and we're _done_ for
> training this hash!). Otherwise (= the other side doesn't know the
> hash == has weight zero on the other side) do the usual: +1 overhere.

yep, differential learning has been discussed quite a quit years ago, and
implemented, though perhaps too naively in crm instead of in LEARN itself.

... 
> additional _technical_ optimization here as you can dispense with two
> CSS files and make it one. But I digress.

yes, that option was already explored. However that's more tricky than it
might seem:

1. experience (testing) with subtraction first (or only) yielded unreliable 
  performance at best: as for me, it meant degraded accuracy till suddenly
  drop to a disaster (spam/ham inverted).
2. likely won't work for a class _per se_: i.e. if you want to assess 
  class-membership of an arbitrary msg.
3. multiclass train/classify needs some (thinking)^2
4. stability problems: the 'filter' ends up with a sharp knee at center, and
  training would behave like what in control theory is iirc a bang-bang 
  controller.
5. your algo above is unstable - indeed, likely for the same reason / 
  pitfall: decrementing works fine as long as we have large counts, but as 
  soon as we reach 0 we've got problems: in such scheme, most bins 
  (commons) would be 0; now train a new msg, which won't trigger any
  decrement but only increment on the intended side; now that side has 
  all those commons +1; now classify almost certainly yields a disater, 
  since also a 'other side' msg will get _big_ score on this side.
  Bottom line: need to track of commons anyway. Perhaps that's just as 
  trivial as not allowing decrement to 0
I've a 'feeling' some trick can be really worked out to make things smaller &
faster & more accurate in that area, too, but till now failed to see
through the fog, and didn't have/spent much time on that, let alone tinker
with C to explore options above.

> I do not 'grok' SVM/SKS/NN/BE at a sufficient level of detail to see
> what I should do there, but suffice to say that they didn't help me
> yet. And, yes, my data feed is HIGHLY unusual and VERY quaint in a

well, depends on data, #classes, resource & time constraints: eg for me
none but the usual OSB*/HS fit my constraints.

> zillion ways and there's a whole _chain_ of issues there that must be
> resolved before you can even consider using a statistical classifier
> _at_ _all_, but suffice to say that it does work well for me with a

ok, but as long as you keep the picture in front of you we see just a nice
brownish cardboard background :)

> The precieved benefit of using statistical / trainable filters for me
> was the idea that it would make the system
> 'auto-tuning'/'auto-adjustable' as I can backpropagate 'knowledge'

what else did you try besides/before crm114?

> into the system in full-auto. Kinda like this: you're SETI listening
> for extraterrestial signals and you just don't know what to look for

ah, ok ... so you're looking for ET ;)

> launch goes well this week, I'll have a bit of time. (please, Lord,
> Jahweh, Allah, Shiva, Maria, and all you other mightily awesomely good
> 'n great fellas and fellettes up (and down) there, care to spare a
> nickle of good fortune on me, please? Thank you kindly, sirs and

eh not that easy, you've to choose whom you want the grace, else they mess 
up ;) 

good luck!

-- 
paolo

PS: I'd rather move such discussions on -discuss or -devel.


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/