Menu
▾
▴
#12 Initial synapse Permanence implies connected synapses
In page 34 of CLA paper at initialization Part
Prior to receiving any inputs, the region is initialized by computing a list of initial potential synapses for each column. This consists of a random set of inputs selected from the input space. Each input is represented by a synapse and assigned a random permanence value. The random permanence values are chosen with two criteria. First, the values are chosen to be in a small range around connectedPerm (the minimum permanence value at which a synapse is considered "connected"). This enables potential synapses to become connected (or disconnected) after a small number of training iterations. Second, each column has a natural center over he input region, and the permanence values have a bias towards this center (they have higher values near the center).
but in Synapse.cs initial permanence is set to ConnectedPerm+0.1f this implies to always connected synapses at initialization.
Also in page 26 of CLA Paper in the definition of Potential synapses:
which is dedicated to all segments both proximal and distal
You are right about the departure from the documentation. I have made some updates of my own based on my experimenting. The +0.1 is meant to give stronger initial connections because I was finding that disconnected synapses have a very hard time becoming connected (very rare), however unused connected synapses are rather easy to trim off and become disconnected later if not helpful. So this is on purpose, and different from the doc due to testing.
Your second question while the doc does say all distal segments have a list of potential synapses, the doc also says later on (during the implementation example) that no such list is actually maintained due to memory concerns. Instead segments+synapses are added from nothing as time goes on to keep memory in check. This is why we initialize to 0 distal segments for all cells and add over time.
From CLA:
I think this area is a good place to look for optimizations later when we get the core CLA stable. Also, random is a CPU hog and in my earlier implementation of CLA I also removed random. If we want to put it back in we should randomize a pool of values around connectedPerm during startup and cycle through them. That would be faster than calling random() function (and synapses creation is in the innermost loop of the CLA and needs to be fast)
Should we leave this ticket open so we don't forget about this?
There are too much things you need to change in order to optimize C# implementation in special it has a huge memory consumption caused by extra repeating class members. Apart from this issue I think It's better to parametrize all constants and semi-constants (variables which will be computed one time and used many times). This way we can easily check and compare them, and at the same time we will need less memory because classes which are constantly or continuously repeating and reallocating (Columns, Cells, Segments, Synapses, ...) will not allocate extra space but just a reference to configurations.
In my C++ Implementation I've made a configuration file and put everything there look at here and here
Dear Softnhard,
did you compare your C++ code with ansiC-version of Barry?
In his version, a lot of improvements have been done.
Best regards
HoaNam
----- Ursprüngliche Nachricht -----
Von: softnhard
Gesendet: 02.04.13 06:05 Uhr
An: [openhtm:tickets]
Betreff: [openhtm:tickets] Re: #12 Initial synapse Permanence implies connected synapses
There are too much things you need to change in order to optimize C# implementation in special it has a huge memory consumption caused by extra repeating class members. Apart from this issue I think It's better to parametrize all constants and semi-constants (variables which will be computed one time and used many times). This way we can easily check and compare them, and at the same time we will need less memory because classes which are constantly or continuously repeating and reallocating (Columns, Cells, Segments, Synapses, ...) will not allocate extra space but just a reference to configurations.
In my C++ Implementation I've made a configuration file and put everything there look at here and here
** [tickets:#12] Initial synapse Permanence implies connected synapses**
Status: open
Created: Mon Apr 01, 2013 01:16 PM UTC by softnhard
Last Updated: Mon Apr 01, 2013 11:24 PM UTC
Owner: Barry Matt
In page 34 of CLA paper at initialization Part
but in Synapse.cs initial permanence is set to ConnectedPerm+0.1f this implies to always connected synapses at initialization.
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/openhtm/tickets/12/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
Related
Bugs: #12
Dear HoaNam
Unfortunately due to fast changes in OpenHTM and at the same time my other task related to my thesis I didn't have enough time to apply barry code improvements on his ansi-C version. But I'll do it after finishing adaptation of my code to OpenHTM.