Was very interesting as far as it seems to be consistent to Jeff's ideas in On Intelligence.
I will try to collect some literature like the one proposed by Doug on Psychology. I wonder how much machine learning algorithms are implemented in this.
Best,
Uwe
Last edit: Uwe Kirschenmann 2013-02-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When I begin read the article, I thought the same..
Maybe they got inspired by HTM and weet deep in this concept. Althought, I think this architecture has been proposed by Vision neuroscientists before Jeff. What Jeff did was extend that cortical archictecture to other applications assuming that cortex is similar in all brain areas.
I didn't read the entire article, but if they use binary representation for synapses, it's too likely they have really based on HTM. I say this because most neural networks academics don't like and don't agree with this approach (binary weights to synapses).
Anyway, I thought very interesting the experiment results with and without top-down feedback on figure 4.
Last edit: David Ragazzi 2013-02-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Some thoughts: Uwe, there is a page on that site that has comparisons of many simulator toolkits. It was interesting to look as some of them and the various approaches, and some ideas are good, but no HTM simulator. They are still stuck in the world of weighted connections / binary representation, like David guessed. The CLA for HTM is based on more detailed understanding of the key biological mechanisim. Jeff / Deliep George and others working with them have distilled the features of sysnaps, dendrite connections and created simple code to model them. With CLA we have a model of how segments of dendrites act as coincidence detectors based on the sum of synapse strength. See pg. 47 of the CLA paper to realize this - there is a nice diagram.
All existing NN models like those listed here http://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators
do not take dendrite processing into account. From the CLA paper pg47: "The biggest change has been in realizing that the dendrites of a neuron are not just conduits to bring inputs to the cell body. We now know the dendrites are complex non-linear processing elements in themselves. The HTM cortical learning algorithms take advantage of these non-linear properties"
In the CLA paper there is a section "recommended reading". Here they recommend the textbook Dendrites, second edition, Stuart, Greg, Spruston, Nelson, Häusser, Michael. They say "This book is a good source on everything about dendrites. Chapter 16 discusses the non-linear properties of dendrite segments used in the HTM CLA" I got this book for my birthday last year and it is excellent. I suggest you get this book if you are interested in the biology or just to spark some new ideas on how to move forward once the basic HTM is understood and software is getting results.
I found only one package out there that simulates a number of different network types and includes simulation of HTM. It also runs on parallel processors. I will try to dig the link up. I ran across it some time ago. I is a university sponsored open source project.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hey Uwe, just found this when looking for that other link. You would be interested in this based on your work and papers that I read that you co-authored (perhaps you already saw this):
Gaze gesture recognition with hierarchical temporal memory networks http://dl.acm.org/citation.cfm?id=2023254
Last edit: Doug King 2013-02-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
well, I found the whitepaper here: http://dspace.mit.edu/bitstream/handle/1721.1/51839/MIT-CSAIL-TR-2010-013.pdf
but if I go the the CNS site I see the simulator does not have the HTM package mentioned in the whitepaper. I remember that it did have an HTM package, but maybe it has been taken out. Or maybe I am not reading the full documentation on the package. Anyway, I think we are on the right track and I need to spend more time with the code :-)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks to Dougs postings I on the bottom-down process i found this very interesting parallel project:
http://grey.colorado.edu/emergent/index.php/Main_Page
Take a look a the screen-shots! Fascinating
There is also a whole book:
http://grey.colorado.edu/CompCogNeuro/index.php/CCNBook/Main
I will try to dig into that a little more
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3376426/
Was very interesting as far as it seems to be consistent to Jeff's ideas in On Intelligence.
I will try to collect some literature like the one proposed by Doug on Psychology. I wonder how much machine learning algorithms are implemented in this.
Best,
Uwe
Last edit: Uwe Kirschenmann 2013-02-21
Oh, here's the original link to CCNbook
When I begin read the article, I thought the same..
Maybe they got inspired by HTM and weet deep in this concept. Althought, I think this architecture has been proposed by Vision neuroscientists before Jeff. What Jeff did was extend that cortical archictecture to other applications assuming that cortex is similar in all brain areas.
I didn't read the entire article, but if they use binary representation for synapses, it's too likely they have really based on HTM. I say this because most neural networks academics don't like and don't agree with this approach (binary weights to synapses).
Anyway, I thought very interesting the experiment results with and without top-down feedback on figure 4.
Last edit: David Ragazzi 2013-02-21
Some thoughts: Uwe, there is a page on that site that has comparisons of many simulator toolkits. It was interesting to look as some of them and the various approaches, and some ideas are good, but no HTM simulator. They are still stuck in the world of weighted connections / binary representation, like David guessed. The CLA for HTM is based on more detailed understanding of the key biological mechanisim. Jeff / Deliep George and others working with them have distilled the features of sysnaps, dendrite connections and created simple code to model them. With CLA we have a model of how segments of dendrites act as coincidence detectors based on the sum of synapse strength. See pg. 47 of the CLA paper to realize this - there is a nice diagram.
All existing NN models like those listed here http://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators
do not take dendrite processing into account. From the CLA paper pg47: "The biggest change has been in realizing that the dendrites of a neuron are not just conduits to bring inputs to the cell body. We now know the dendrites are complex non-linear processing elements in themselves. The HTM cortical learning algorithms take advantage of these non-linear properties"
In the CLA paper there is a section "recommended reading". Here they recommend the textbook Dendrites, second edition, Stuart, Greg, Spruston, Nelson, Häusser, Michael. They say "This book is a good source on everything about dendrites. Chapter 16 discusses the non-linear properties of dendrite segments used in the HTM CLA" I got this book for my birthday last year and it is excellent. I suggest you get this book if you are interested in the biology or just to spark some new ideas on how to move forward once the basic HTM is understood and software is getting results.
I found only one package out there that simulates a number of different network types and includes simulation of HTM. It also runs on parallel processors. I will try to dig the link up. I ran across it some time ago. I is a university sponsored open source project.
Hey Uwe, just found this when looking for that other link. You would be interested in this based on your work and papers that I read that you co-authored (perhaps you already saw this):
Gaze gesture recognition with hierarchical temporal memory networks
http://dl.acm.org/citation.cfm?id=2023254
Last edit: Doug King 2013-02-21
well, I found the whitepaper here: http://dspace.mit.edu/bitstream/handle/1721.1/51839/MIT-CSAIL-TR-2010-013.pdf
but if I go the the CNS site I see the simulator does not have the HTM package mentioned in the whitepaper. I remember that it did have an HTM package, but maybe it has been taken out. Or maybe I am not reading the full documentation on the package. Anyway, I think we are on the right track and I need to spend more time with the code :-)