Questions about OpenHTM implementation

[Michael Ferrier]

Hi,

I'm new to HTM and I'm looking to implement CLA using C++. I'm modeling my implementation after the OpenHTM implementation. I want to complement the coders on a very clearly written program that is a pleasure to read through, and potentially a great accelerator for the HTM community! After going through the code I do have just a few questions that I'm hoping someone might be able to help me with.

1) The biggest difference between the Numenta white paper and the OpenHTM implementation appears to be that Numenta would differentiate only between sequence segments (that predict activation in the next time step) and non sequence segments (that predict activation some indeterminate time after the next time step). OpenHTM on the other hand records a NumPredictionSteps value in each sequence segment, and a segment is only updated if this value matches, and so it differentiates between, say, a segment predicting activation at time t+8 vs time t+9.

This seems like a major change, and I'm wondering what motivated it? Is there a discussion online anywhere of the pros and cons of the two approaches? I had no luck searching the old Numenta forum for an answer.

In a realtime application, where, for example, the same note in a melody to be identified or the same phoneme in a word to be identified may have slightly different durations each time it's encountered, wouldn't OpenHTM's method of differentiating between different NumPredictionSteps values for a segment cause these slight changes in duration to result in completely different representations of e.g. melodies or words that differ only slightly in the durations of their parts? Or am I misunderstanding this?

2) In the Column method PerformBoosting(), why is Boost multiplied by 1.05 (rather than 1.0) when the ActiveDutyCycle equals minDutyCycle? If (ActiveDutyCycle < minDutyCycle), Boost is set to (minDutyCycle / ActiveDutyCycle), and so as ActiveDutyCycle appraoches minDutyCycle, Boost approaches 1.0, but then if they are equal, Boost is multiplied by 1.05, causing a discontinuity, before Boost returns to 1.0 when (ActiveDutyCycle > minDutyCycle). I wonder what the justification is for this? Is it common that ActiveDutyCycle will exactly equal minDutyCycle, and in that situation was it found useful to increase Boost until ActiveDutyCycle exceeds minDutyCycle?

3) In Synapse.cs, PermanenceDec = 1.01, while PermanenceInc = 1.015. I'm wondering why permanence values are increased 50% faster than they are decreased?

Again, kudos on the great work! There are just these few points that I'd like to understand better.

Thanks,

-Mike

[Barry Matt]

Hi Mike, great questions!

1)
Recording the exact prediction steps for a segment is an obvious change, here's why. Far from a major change, it has no impact whatsoever on the algorithm itself as code can still ask for sequence (t=1) versus non-sequence (t>1). So there is no change there. Additionally there is no memory or performance cost for doing this, as we need only store a single number per segment (were doing that anyway to store the sequence vs non-sequence flag), and performance is merely t==1 as equivalent to isSequence==true.

The biggest difference though is now we have a lot more detailed information about the predictions at our disposal. We can now update our UI to display exactly how far away predictions are being made for within each cell. This is a much more helpful display than simply seeing a large mass of cells predicted for "sometime t>1".

The user could now ask for things like "tell me when a particular prediction is 5 steps away". This could provide warning further in advance for things about to happen soon but not immediately. We also can provide HTM parameters for limiting the maximum prediction steps. So a developer can say "only store predictions for things at most 10 time steps away". Without storing the number there is no way to limit this.

So really adding the NumPredictionSteps is a very obvious change with no downside.

2)
The boost value is multiplier so if it equals 1.0 it means the boost will have no effect. If we want boosting we want it to have some positive so the 1.05 is simply a way of saying if the boost is down to 1.0 (no effect) then force it to be something positive so at least some boost will occur. That's really the only reason for it.

3)
You are right they are different on purpose. My first HTM implementation was over 2 years ago and in that time my experiments have found that it is a lot easier for permanence to go down than it is for it to go up. Generally when seeing something for the first time it was not good to have a single negative example cancel out the initial learning. More often than not I was encountering negative examples far before seeing repeat positive examples. Instead if positive learning is larger it means we need to see several negatives to erase it (rather than just one) and this helped retain initial learning long enough to see repeats for further strengthening. It is still so far an open value as experiments continue from here. These are just my findings so far regarding a good default.

Barry

[David Ragazzi]

Hi Barry,

The biggest difference though is now we have a lot more detailed information about the predictions at our disposal. We can now update our UI to display exactly how far away predictions are being made for within each cell. This is a much more helpful display than simply seeing a large mass of cells predicted for "sometime t>1"

Thanks to your insight, I could add a visualizer for each prediction step in Main form. However I also have a question. :-)

We know which columns will be active in T=3 because NumberPredictionSteps for cells segment is 3. But what if this column ALSO will be active in T=2? Should I include the segments that will be active only when T=3 or should I include the segments that are active in T=2 too, ie NumberPredictionSteps <= 3 instead NumberPredictionSteps = 3?

  // Get the representation for each T time interval
  int[,] predictionsBitMap = htmRegion.GetPredictingColumns();
  for (int t = 1; t <= 3; t++)
  {
    int[,] tPredictionBitMap = new int[predictionsBitMap.GetUpperBound(0) + 1, predictionsBitMap.GetUpperBound(1) + 1];
    for (int x = 0; x <= predictionsBitMap.GetUpperBound(0); x++)
    {
      for (int y = 0; y <= predictionsBitMap.GetUpperBound(1); y++)
      {
        // If current column is within prediction limit

        //if (predictionsBitMap[x, y]> 0 && predictionsBitMap[x, y] <= t) or:
        if (predictionsBitMap[x, y] == t)
        {
          tPredictionBitMap[x, y] = 1;
        }
      }
    }

    // Draw the prediction in T time
    ....
  }

In the case of I misunderstood this feature, please could you explain these datails in order I improve the UI? :-)

David

[Barry Matt]

Well in my Java UI I only have it set to show the earliest prediction for each cell. So if a cell is predicted for both t=2 and t=3 then the UI only show the soonest, which is t=2. You can see the GetPredictingColumns is returning the soonest prediction value for each cell so your predictionsBitMap will contain t=2 for that cell in question. In my Java UI I simply colored each cell based on its soonest prediction value (as directly returned from GetPredictingColumns), so t=1 would be green, t=2 would be yellow, t=3 is blue, etc.

This is just one way to display things, we of course can do things differently if something else makes more sense. But if you want to just copy what I was doing you would not need any of your code above, you can simply call GetPredictingColumns and those numbers would directly decide how cells are colored, no additional work.

If you have an idea for a different display that is fine too. If we did want to somehow show all the predictions (t=2 and t=3 for example) we would need a new helper method in Region to return all the values rather than simply the soonest one. This is easy to add if you would like this option too.

[Michael Ferrier]

All makes sense now -- thanks Barry for taking the time to answer my questions!

-Mike

[Michael Ferrier]

Another question for you Barry (or for whoever might know the answer):

In the Column method ComputeOverlap(), overlapTemp is rounded to precision 2, and then rounded again after being multiplied by Boost. It's not clear to me why this is being done. It would serve to clump together the overlap values of Columns with similar overlap values, but I'm not sure why this would be useful? And why rounded twice?

Thanks,

-Mike

[David Ragazzi]

Hi Michael,

Have you push the last project changes by Git? I tried find this code but without success.

Bellow the current Column.ComputeOverlap() method:

internal void ComputeOverlap()
        {
            // Calculate number of active synapses on the proximal segment
            this.ProximalSegment.ProcessSegment();

            // Find "overlap", that is the current number of active and connnected synapses
            int overlap = this.ProximalSegment.GetNumberActiveSynapses(true);
            if (overlap < this._minOverlap)
            {
                overlap = 0;
            }
            else
            {
                overlap = (int)Math.Round(((float)overlap) * Boost);
            }

            this.Overlap = overlap;
        }

[Michael Ferrier]

Hi David,

Thanks a lot, my code was a few days old. The new version is much clearer.

-Mike

[Michael Ferrier]

Found an issue in GetMaxDutyCycle() which may qualify as a minor bug.

The nested loop finds the max among the _activeDutyCycle values of all columns within inhibitionRadius of this column. But the loops run from e.g. (positionX - inhibitionRadius) through (positionX + inhibitionRadius - 1), rather than through (positionX + inhbitionRadius), because the 'for' loops use < rather than <=. This makes the area that's scanned asymmetrical around the column. Just a minor issue, probably not causing problems, but I thought it might be worth pointing out.

[Barry Matt]

Yes the for loops use < rather than <=. Did you notice these lines:

finalX = Math.Min(Region.Width, finalX + 1); //extra 1's for correct looping
finalY = Math.Min(Region.Height, finalY + 1);

An extra 1 is added to compensate for the <. It also makes sure to cut off at the edges of the region. Are you referring to something else? Regardless this is indeed a minor issue related to boosting (which we have not really been using so far). Even if a slight bit of radius were trimmed off it would hardly make much difference in the results.

However I do appreciate your attention to detail in the code. Keep the feedback coming!

[Michael Ferrier]

Another question. In Column's ComputeOverlap(), a few weeks ago the determined overlap number was then being divided by the area of the input to that Column. This had the result of making the overlap numbers for all columns comparable, including those in the middle of a Region with large receptive fields, and those at the edge of a Region with smaller receptive fields due to being cut off at the edge of the Region. However in the latest versions of the code that division was removed (though the comment still reflects the previous way of doing it). Wouldn't removing that division go back to giving the edge columns an unfair disadvantage in inhibition, since they would tend to have lower absolute overlap, since they have smaller receptive fields? Is this problem now being solved in some other way?

Thanks,

-Mike

[Barry Matt]

This is a good point. You are right, but only in cases where a locality radius is in effect. By default the region will use locality radius = 0 which means columns can connect to any bits from the input. In that case all columns will have an approximately equal number of bit connections from the same area thus the direct overlap comparison works.

Locality radius was something I added in my initial implementation but not mentioned in the Numenta doc (which only described the full connection case).

Anyway I will look into restoring the percentage-based overlap when locality radius is enabled as you right it would be more accurate for that situation.

[Michael Ferrier]

Hi Barry, thanks for all the comments! I did some testing with this issue and found that simply dividing a column's overlap by the number of input bits (like the way it used to work) introduces a problem. Doing this makes the overlap number much smaller. That works fine for purposes of comparing different columns' overlap values for inhibition in IsWithinKthScore(), but it doesn't work well when the overlap values (via overlapDutyCycle) are compared against minDutyCycle to determine whether a column's proximal synapse permanence values should be boosted. If overlap was divided by the number of inputs and so made much smaller, it's no longer on the same "scale" as activeDutyCycle and minDutyCycle, and so it's much more likely to cause (unnecessary) boosting of proximal synapse values.

So probably simply dividing overlap by the number of inputs in ComputeOverlap() isn't a good solution for edge columns. One possible alternative would be to store overlap without doing that division, and then divide by the number of inputs "only the fly" only in IsWithinKthScore(), when comparing the overlap values of two columns.

An alternative might just be to let Boost do its job; it seems to me that if edge columns are active substantially less often, Boost should effectively increase their overlap values on the fly and make up for their competitive disadvantage of having fewer inputs bits than non-edge unit. I haven't tested this though.

[Michael Ferrier]

Found another possible issue. In Segment's AdaptSynapses(), a synapse's permanence is only increased if that synapse is both active and connected. The Numenta pseudocode (line 21) only requires that it be active. By also requiring that it be connected, an unconnected proximal synapse can never have its permanence increased and so can never become connected. Is this a bug, or is there a reason for doing it this way?

[Barry Matt]

Good catch! AdaptSynapses() is only used in the spatial pooler, but you are definitely right it should require active only regardless of connection otherwise disconnected synapses will never come back. This might explain a few problems I have been seeing in the spatial pooler for some time now. Change has been committed.

[Michael Ferrier]

Region's AverageReceptiveFieldSize() currently determines the average distance from center of each connected proximal synapse in the Region. The Numenta docs describe how it should work it this way:

"The radius of the average connected receptive field size of all the columns. The connected receptive field size of a column includes only the connected synapses (those with permanence values >= connectedPerm). This is used to determine the extent of lateral inhibition between columns."

This description is ambiguous, but I would think that a column's receptive field radius would refer to the distance of the connected proximal synapse with the greatest distance from center, rather than to the average of the distances of all connected proximal synapses (which would result in a receptive field radius that leaves out roughly half of the connected synapses). I'm interested in how others interpret how this should work.

[Doug King]

I believe because this radius " is used to determine the extent of lateral inhibition between columns" we need to think of this as the inhibitor radius. So maybe it is not good to have the inhibition = max radius, but some smaller percentage of that.

I would like to add a setting that allows us to manipulate this percentage to see what happens, but I suspect Numenta found that the average works ok.

[Michael Ferrier]

I found what may be a problem with the temporal pooler, unless I'm misunderstanding it. Segment's list of ActiveSynapses, compiled in ProcessSegment(), contains only those synapses that are both active and connected. The ActiveSynapses list is then used by UpdateSegmentActiveSynapses(), which passes it to the SegmentUpdate() constructor as the list of active synapses to be incremented later (if the unit actually becomes active). The problem is, this list contains only the synapses that are both active and connected, so synapses to cells that are active never have the opportunity to become connected, if they are not connected already. In my tests, often segments that should become active never gain enough active, connected synapses to become active, because of this problem.

The relevant part of the Numenta doc, for getSegmentActiveSynapses(), states: "Let activeSynapses be the list of active synapses where the originating cells have their activeState output = 1 at time step t." It doesn't mention any requirement that they also be connected synapses.

It seems to me that fixing this would require changing ProcessSegment() so that it fills ActiveSynapses with the list of all active synapses, not just those that are connected, and looks at how many are connected only to determine the value of IsActive. The methods GetActiveSynapseCount() and GetPrevActiveSynapseCount() would then also need to be changed accordingly.

Does this make sense? Or am I misunderstanding how this should work?

[Nick]

It feels like I saw similar opinion somewhere here before, I'll try to find it...
Why don't you try changing that yourself and testing?

[Nick]

That was your post too :)
I don't know why it's there but found this in Uwe's UIHtmCal code which is initial version of this project:

//Override properties from Synapse
        public  override bool isActive
        {
            get
            {
                return inputSource.IsActive && isConnected;
            }
        }

[Nick]

I've tried and 'connected' performance is better for me in ABBCBB and jumping ball tests.
In Numenta doc it writes:

For every potential synapse on the active dendrite segment, increase the permanence of those synapses that are connected to active cells and decrement the permanence of those synapses connected to inactive cells.

Yes, the word 'connected' is ambiguous here.

[Michael Ferrier]

Thanks for the replies Nick. Yes there was a similar issue in the spatial pooler, where AdaptSynapses() was only increasing a synapse's permanence if that synapse was already connected.

I think this issue with the temporal pooler mostly can become a problem if the spatial pooler is being used. This is because the issue doesn't affect newly created segments, but it does damage the ability of an existing segment to adapt to represent an altered set of inputs. In my tests, that would happen when boosting would cause the spatial pooler to change the sparse representation of individual letters over time. If HardcodedSpatial is used, however, that never happens and so this problem might not show up.

I made the changes in my C++ implementation, to include all active synapses in ActiveSynapses, not just those that are connected. After that change, it worked much better for me, with performance no longer degrading over time as boosting gradually changed the sparse representations of the inputs. I tried the ABBCBB sequence and by time 30 it was successfully predicting all activations. That wasn't the case for your ABBCBB test Nick?

I made these changes to my C++ code so it doesn't directly translate to the C# codebase, but I'll paste the changes I made below in case it would be helpful, since it seemed to work well:

void Segment::ProcessSegment()
{
  ActiveConnectedSynapsesCount = 0;

  ActiveSynapses.Clear();

  FastListIter synapses_iter(Synapses);
  for (Synapse *syn = (Synapse*)(synapses_iter.Reset()); syn != NULL; syn = (Synapse*)(synapses_iter.Advance()))
  {
    if (syn->GetIsActive())
    {
      ActiveSynapses.InsertAtEnd(syn);

      if (syn->GetIsConnected()) {
        ActiveConnectedSynapsesCount++;
      }
    }
  }

  IsActive = (ActiveConnectedSynapsesCount >= ActiveThreshold);
}

int Segment::GetActiveSynapseCount()
{
  return ActiveSynapses.Count();
}

int Segment::GetActiveConnectedSynapseCount()
{
  return ActiveConnectedSynapsesCount;
}

int Segment::GetPrevActiveSynapseCount()
{
  return PrevActiveSynapses.Count();
}

I also changed the 5 places in Cell and Column code where GetActiveSynapseCount() and GetPrevActiveSynapseCount() are called, to accommodate the changes to those functions.

[Nick]

I misused this line last time:

IsActive = (ActiveConnectedSynapsesCount >= ActiveThreshold);

.

I tried the ABBCBB sequence and by time 30 it was successfully predicting all activations. That wasn't the case for your ABBCBB test Nick?

Now it works with the same result as with isConnected, yes on step 30 it learns correct sequence.

I think this issue with the temporal pooler mostly can become a problem if the spatial pooler is being used.

So when we will make some tests with SP we should see this.
Thanks for advices, I might need some expert approval to commit the changes.

[Barry Matt]

Hi Michael,
Once again nice catch. You are right again I believe, the updates should happen if active regardless of connection. For the temporal pooler case this was a bit less obvious since we start with no segments and add them over time (with all new synapses that start already connected). So this would only be a problem if some synapses were later decremented to become disconnected. Then once disconnected they would never come back, which is not what we want.

I will see about making the necessary changes to fix the problem. Thanks for the close look!

[Nick]

Do we need Segment.GetNumberPreviousActiveSynapses() method now? It's used with false everywhere so as GetNumberActiveSynapses is mostly used with 'false'.