Re: crm filter cleanup on productive system

Hi Bill,

thanks for that explaination ! but i really do hard to understand it,
maybe you have some minutes to go through it.

> > Now, as to how a neural net exactly works, that's still the subject of
> > PhD theses, but the general idea is this:
> >
> > 1) you have a retina: in the CRM114 implementation this is 8192 slots
> > (called the "retina") which each get the number of occurrences that a
> > 32-bit OSB hash feature occurred that _just happened_ to have the low
> > order 13 bits equal to the slot number (this number is likely to
> > change in the future, and the smallness of this number compared is one
> > reason why large texts tend to break the neural net; when all of the
> > slots have equal counts, there's nothing to learn!).

I understood it like this:
We have 8192 starting points called a retina. Now we take a sentence or
question or just any data input, split it up and for each unit we
generate a 32bit OSB hash.

Now i dont understand this part with the low order 13 bits equal to the
slot number.

> > 2) You then have a bunch (8 or 16, also likely to change) of summing
> > junctions (called "first layer neurons) which each have an independent
> > weight for all 8192 input slots (that's 64K input weights).  The
> > output of these summing junctions then is mapped by a nonlinear
> > function called a sigmoid, which looks like a tan(x) curve rotated
> > about the 45-degree line.

After that retina is filled with the data, we connect each slot of these
8192 with 8 or 16 junctions, we build a kind of network here. Each
connection has an own weight, at the beginning i assume the weight is
randomized to give it any reasonable start. The "first layer neurons"
are the first stage of "hidden units" and the 8192 retina slots are the
input units right ? Now how and where do you apply the first sigmoid
function ? You take all the data coming from the retina weight it by the
weight for the given path and put it together with the sigmoid function ?


> > 3) Then you have a second layer of (8 or 16) summing-junction neurons;
> > these each have one independent weight for each sigmoid from the first
> > layer (for 64, 128, or 256 independent weights).  Those summed outputs
> > then also get sigmoided.

here we have another layer of hidden units again we apply new weights
per path and sigmoid them again.

> > 4) One more layer, with just two summing-junction neurons and sigmoids
> > gives us the output - the [0]th neuron is the "in class" signal, and
> > the [1]th neuron gives the "out of class" signal.

if i understand right these are the output units, which in this case can
produce yes or no.

> > Training is a bit harried; you have to do a gradient descent from
> > the desired output to the mapped inputs; there's some calculus
> > involved, as well as a bit of luck (convergence is NOT gauranteed!)
> > You also have to initialize with small random numbers because
> > otherwise, the standard training algorithms will be "stuck on center"
> > forever and you get no training at all.

it would be nice if you could also explain this if you got some
freetime, i know its complicated

What kind of network have you implemented ? i feel its a kind of pattern
associator network with 2 extra hidden layers. therefore for learning
you use some kind of delta or hebb rule learning ?

Thanks

Frank

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H

_______________________________________________
Crm114-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crm114-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)

iEYEARECAAYFAkmhtcEACgkQI0IqF/J+ZQ/7nACdGVyLEgTPITTyGcHpn/Acfyy7
HMMAn3WsSbIIsGCXu3oSO1lwQN4D0qdN
=mvGi
-----END PGP SIGNATURE-----