I've been thinking about hierarchy and feedback from higher region.
I thought : "What is the purpose of feedback?", what kind of information could be there?
The only answer I found is the correction of the lower region.
I still don't know how this will work but I have some very simple idea that includes actual mechanics.
The lower regions record very simple transformations. If they are presented with a transformation, in most cases I think, you can not expect it to be true without higher meaning.
Even further, the lower region has it's own limited memory capacity.
It can't record all the transformations in the world for every different scenario. I think there must be some higher meaning from higher region who will correct what the lower region predicts.
This is the plan I have, let's say we have two regions :
A and B. A is the lower region and B is the higher region.
step 1 : A is given an initial input
step 2 : A predicts
step 3 : B takes A total input (initial + prediction - and even few predicting few steps forward in time, not just one)
step 4 : B predicts "Mix". (Mix is the prediction of the combined initial B inputs, which are A's initial and A's prediction)
step 5 : B gives A "Mix" in order to correct A's prediction
step 6 : A predicts correctly and gives initial input to B.
but as I see, there are too many steps - and too many places to store temporary results, and what will we learn from them.
so I focus on 3 steps and will think about the rest later :
step 1 : A is given an initial input
step 2 : B gives A "Mix" in order to correct A's prediction (Maybe "B" mix will come from the previous timestep, need some more thought here)
step 3 : A predicts correctly and gives initial input to B.
The "correction" step - the most complex step :
The way I think about "correction" is like a "modifier", a "key", a "context" to the input.
we already know that we use "contexes" as cells in columns, so I thought the "correction" modifier could be another cell which is turned on or off depending on whether we are correcting or not.
But then I thought, this modifier must be stronger than a regular one cell, which might be ignored when activating a segment as part of a group.
so what I propose is two mini-columns per one single column.
one is for "non-corrected input", and the other is for "corrected" input.
I still do not know the exact details about the interaction between the two columns.
When we receieve "B Mix" input from the higher region, we look at the input and
activate or de-activate the right column.
Those are my thoughts so far, many many questions arise from this concept. and I will continue to think about them.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've been thinking about hierarchy and feedback from higher region.
I thought : "What is the purpose of feedback?", what kind of information could be there?
The only answer I found is the correction of the lower region.
I still don't know how this will work but I have some very simple idea that includes actual mechanics.
The lower regions record very simple transformations. If they are presented with a transformation, in most cases I think, you can not expect it to be true without higher meaning.
Even further, the lower region has it's own limited memory capacity.
It can't record all the transformations in the world for every different scenario. I think there must be some higher meaning from higher region who will correct what the lower region predicts.
This is the plan I have, let's say we have two regions :
A and B. A is the lower region and B is the higher region.
step 1 : A is given an initial input
step 2 : A predicts
step 3 : B takes A total input (initial + prediction - and even few predicting few steps forward in time, not just one)
step 4 : B predicts "Mix". (Mix is the prediction of the combined initial B inputs, which are A's initial and A's prediction)
step 5 : B gives A "Mix" in order to correct A's prediction
step 6 : A predicts correctly and gives initial input to B.
but as I see, there are too many steps - and too many places to store temporary results, and what will we learn from them.
so I focus on 3 steps and will think about the rest later :
step 1 : A is given an initial input
step 2 : B gives A "Mix" in order to correct A's prediction (Maybe "B" mix will come from the previous timestep, need some more thought here)
step 3 : A predicts correctly and gives initial input to B.
The "correction" step - the most complex step :
The way I think about "correction" is like a "modifier", a "key", a "context" to the input.
we already know that we use "contexes" as cells in columns, so I thought the "correction" modifier could be another cell which is turned on or off depending on whether we are correcting or not.
But then I thought, this modifier must be stronger than a regular one cell, which might be ignored when activating a segment as part of a group.
so what I propose is two mini-columns per one single column.
one is for "non-corrected input", and the other is for "corrected" input.
I still do not know the exact details about the interaction between the two columns.
When we receieve "B Mix" input from the higher region, we look at the input and
activate or de-activate the right column.
Those are my thoughts so far, many many questions arise from this concept. and I will continue to think about them.