CLA and HTM are fundamentally different from other algorithms like neural networks or hidden markov models. The math and learning mechanisms are similar, but the premise requires a careful approach in how you create systems.
For example, as evidenced in a previous thread, if you tried to teach a region math by giving it 1+0 through 1+4 , and then did something like A+0, you'd end up with a prediction of 4, because the A symbol resembles 4. There's no context to the input, and it has no way to place the data in a relevant sequence or pattern.
The trick is finding a sequence of patterns that results in the logical operation you're trying for. I'd equate this to implementing the basic operations of assembly language, or doing binary operations on arbitrary numbers. This could happen naively, through a logical progression of training on simple math concepts (think of teaching a child to add) or it could be done manually, to much better effect, by exploring the development of regions being trained manually until a systematic approach becomes apparent.
So on that note, I think it would be worthwhile to explore some basic concepts in HTM. I'd start with outlining the temporal capacity for a single column, and how each cell affects the predictive quality.
I'd want to do something like 100 different test configurations, starting with a simple on/off pattern (might just require 2 cells) and moving up to a nonlinear sequence of 1000 steps.
After doing a single cell, then I'd work in 2,3,4 and so on, up to 10.
This would give us a formula to empirically determine the capacity for temporal complexity of a given network.
Similarly, I'd want to measure the capacity for spatial complexity. This means measuring the accuracy of pattern recognition for a given sparsity against the dimensions of the input space. What this means is that the more complex your input space, the less sparse your representations. There may be a sweet spot around the 20% mark as evidenced by Numenta's research.
It should be possible to estimate the temporal and spatial complexity of a given problem, and tailor the region to the task - perhaps a region could be developed that performs the task for us (metalearning!)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It is important to think of HTM as a pattern learning and predicting engine and one that is best suited to time based patterns that have data-points close together in time, i.e. temperature changes, motion tracking, etc.
Learning math is such a high level abstraction that it is not appropriate application for the software we have now.
So conforming your data to something that is 'natural' for HTM input is part of solving the problem if you do not already have data that is a good fit. For example, if you want to train for speech recognition you would not feed the raw audio waveform to the HTM. Our ears pre-process complex wave forms into thousands of discrete envelopes of single frequencies via the hairs in the inner ear. So the brain gets a few thousand inputs of varying amplitude and then it goes to work on correlating those signals into recognizable patterns. So we would want to do the same, convert audio via FFT to many filtered frequency bands that have signals with data-points close together as the signal varies over time.
All problems should be viewed this way, stock market predictions, etc.
A simple rule of thumb: the easier it is for you to visualize a signal on a graph, the better suited it is for input to the HTM.
Last edit: Doug King 2013-04-09
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Consider the way you'd do math on an abacus, or with a slide rule; there are ways of using patterns and predictions to do mathematics, because the prediction would result in an implicit calculation of the input pattern - the region/s need to account for that type of model, and not frame them in a way that imposes some sort of shorthand context, like the way we use visual symbols.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I agree with Doug: Math is on a level of high abstraction. You must first learn different concepts (as in psychology it is said: object permanence and different stages as proposed by f.ex. jean piaget). the application of mathematics in humans needs the flexible combination of learned categories and symbolic transformations. it is very different from a turing machine that is the result of a human mind on how to best transform or operate on symbols. the idea of htm is not to simulate or program a new turing machine, but to understand the different operations of the mind machine :-)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
CLA and HTM are fundamentally different from other algorithms like neural networks or hidden markov models. The math and learning mechanisms are similar, but the premise requires a careful approach in how you create systems.
For example, as evidenced in a previous thread, if you tried to teach a region math by giving it 1+0 through 1+4 , and then did something like A+0, you'd end up with a prediction of 4, because the A symbol resembles 4. There's no context to the input, and it has no way to place the data in a relevant sequence or pattern.
The trick is finding a sequence of patterns that results in the logical operation you're trying for. I'd equate this to implementing the basic operations of assembly language, or doing binary operations on arbitrary numbers. This could happen naively, through a logical progression of training on simple math concepts (think of teaching a child to add) or it could be done manually, to much better effect, by exploring the development of regions being trained manually until a systematic approach becomes apparent.
So on that note, I think it would be worthwhile to explore some basic concepts in HTM. I'd start with outlining the temporal capacity for a single column, and how each cell affects the predictive quality.
I'd want to do something like 100 different test configurations, starting with a simple on/off pattern (might just require 2 cells) and moving up to a nonlinear sequence of 1000 steps.
After doing a single cell, then I'd work in 2,3,4 and so on, up to 10.
This would give us a formula to empirically determine the capacity for temporal complexity of a given network.
Similarly, I'd want to measure the capacity for spatial complexity. This means measuring the accuracy of pattern recognition for a given sparsity against the dimensions of the input space. What this means is that the more complex your input space, the less sparse your representations. There may be a sweet spot around the 20% mark as evidenced by Numenta's research.
It should be possible to estimate the temporal and spatial complexity of a given problem, and tailor the region to the task - perhaps a region could be developed that performs the task for us (metalearning!)
It is important to think of HTM as a pattern learning and predicting engine and one that is best suited to time based patterns that have data-points close together in time, i.e. temperature changes, motion tracking, etc.
Learning math is such a high level abstraction that it is not appropriate application for the software we have now.
So conforming your data to something that is 'natural' for HTM input is part of solving the problem if you do not already have data that is a good fit. For example, if you want to train for speech recognition you would not feed the raw audio waveform to the HTM. Our ears pre-process complex wave forms into thousands of discrete envelopes of single frequencies via the hairs in the inner ear. So the brain gets a few thousand inputs of varying amplitude and then it goes to work on correlating those signals into recognizable patterns. So we would want to do the same, convert audio via FFT to many filtered frequency bands that have signals with data-points close together as the signal varies over time.
All problems should be viewed this way, stock market predictions, etc.
A simple rule of thumb: the easier it is for you to visualize a signal on a graph, the better suited it is for input to the HTM.
Last edit: Doug King 2013-04-09
Consider the way you'd do math on an abacus, or with a slide rule; there are ways of using patterns and predictions to do mathematics, because the prediction would result in an implicit calculation of the input pattern - the region/s need to account for that type of model, and not frame them in a way that imposes some sort of shorthand context, like the way we use visual symbols.
I agree with Doug: Math is on a level of high abstraction. You must first learn different concepts (as in psychology it is said: object permanence and different stages as proposed by f.ex. jean piaget). the application of mathematics in humans needs the flexible combination of learned categories and symbolic transformations. it is very different from a turing machine that is the result of a human mind on how to best transform or operate on symbols. the idea of htm is not to simulate or program a new turing machine, but to understand the different operations of the mind machine :-)