Re: crm filter cleanup on productive system

Bill Yerazunis wrote:

> Consider, right now a feature is really of the form xxxx[0/1]NNN,
> where the first 19 bits are _thrown away_.  Thus, any pair of 
> features that differ only in the first 19 bits will not be separable
> by any neuron, anywhere.
> 
> The fix is to keep the features separated.  Ideally, we'd have the 
> retina be 2^32 features long, but that would simply not work at all
> on X86, and work very slowly on X64.  
> 
> So, I'm working on methods such as:
> 
>     retina_slot = feature_hash MOD ( retina_size - neuron_number)
> 
> (and other ideas!) to cause a particular 32-bit feature to end up
> on a different retina slot for each input neuron.

Pardon my intrusion, I am definitely not qualified to reason about these math
subtleties, but I'll try anyway.

The attempt to get "true 32-bit significance" seems unfounded to me. What is
going on is that original arbitrary size "pieces of data" (words, whatever) are
"folded" into short hashes, that *happen to be* 32bit in this particular
implementation. When you use 13 bits of the 32bit hash for the NN, you do not
"loose 19 bits", you just fold the original data into a shorter hash.

Obviously, the shorter the hash, the higher is probability of "hash clash", i.e.
of different original pieces of data being converted into the same hash value.
This probability exists for 32bit hashes as well as for 13bit, it's just way
lower for 32bit.

Now, when you are trying to "cheat" like "retina_slot = feature_hash MOD (
retina_size - neuron_number)", you are just changing the "overall" hash function
that converts the original data into 13bit hash (in this case, roughly, by
taking different 13 bits from the "fist stage" 32bit hash), but it does not
change the probability of different original pieces ending up with the same
hash. (Well, in assumption that the initial hash function is cryptographically
sound.) What you achieve is just a different hash function that packs into the
same size hash. The probability of two different "original features" having the
same hash will not change.

IANAM ;-)

Eugene

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H

_______________________________________________
Crm114-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crm114-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJo5B+frw/cIw6UWkRAl4hAKChMGbH1bqqKEmwmbF3NST1+7g3ggCgqoR2
H/o3EK23bwsLddHPbZRrBzk=
=fsu8
-----END PGP SIGNATURE-----