Hi, I've recently found out about HTMs and they seem very interesting. I'm just an amateur programmer and I have no knowledge of statistical models but I'm determined to understand how this theory works :)
I've read the Numenta documentation and looked at your project and I started to write my own implementation of it to better understand this. I have some questions I need help with.
When initializing region we create proximal synapses to a random set of inputs for each column. Is it possible that with this randomness some input bits don't have any synapses connected to them and does it matter if so?
Numenta documentation says each column has a natural center over the input and synapses in the center have a higher permanence when initializing but it does not say by how much, why is it necessary and how I should calculate this?
When permanence of distal synapses falls to zero should we delete them? I don't see anything mentioned about this in the documentation.
Numenta documentation talks about Xth input bit but then it also talks about center of input. Is the input supposed to be a matrix or just a series of bits and does it matter which?
How big should a region be compared to input and how many regions are needed? I know that larger regions means more complex patterns and more regions means almost the same but is a region supposed to be larger than input or smaller and how many regions is needed to solve some usual problems?
When making predictions how do I know how far ahead in timesteps is the prediction? Also can I calculate the probability of the prediction based on past events?
How do I convert text or images to input that makes sense for region? With images I mean not black and white dots but let's say gray or even colored. With text do I have to give input one letter at a time (because the input size is fixed)?
I didn't see anything written about naming patterns in the documentation. How do I know which set of columns correspond to a pattern I want to know about? Do I have to tell the region that current input contains patterns "A", "B" and "C", then later I tell ok now this input contains "B" and "C" and it compares with previous input to find the common columns so it can narrow down which columns correspond to which names? If that is true then how would I do this and if I have multiple regions in layers which region do I tell this and which region should I later ask about it? Also is it possible with boosting and that stuff that columns that previously corresponded to "A" now correspond to something else or "A" is now different columns?
Is there a more thorough documentation than Numenta's on HTMs that I could read?
Thank you
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Q
When initializing region we create proximal synapses to a random set of inputs for each column. Is it possible that with this randomness some input bits don't have any synapses connected to them and does it matter if so?
A
doesnt matter, if the synapse completely disconnects, you rerandomize it to read the current pattern coming in.
Q
Numenta documentation says each column has a natural center over the input and synapses in the center have a higher permanence when initializing but it does not say by how much, why is it necessary and how I should calculate this?
A
I say just randomize it completely, then youll be sharing the space completely between symantics, and they all produce bit clouds, in the right combination they should make an image. you will probably find with pure random youll be making distinct sizes and shapes anyway.
Q
When permanence of distal synapses falls to zero should we delete them? I don't see anything mentioned about this in the documentation.
Numenta documentation talks about Xth input bit but then it also talks about center of input. Is the input supposed to be a matrix or just a series of bits and does it matter which?
A
when a distal synapse falls to zero you rerandomize its location to be a part of the entering data pattern, so its always got a chance of coming back to reuse.
Q
How big should a region be compared to input and how many regions are needed? I know that larger regions means more complex patterns and more regions means almost the same but is a region supposed to be larger than input or smaller and how many regions is needed to solve some usual problems?
A
I say equal size to data complexity, if its 64x64 1 bit pixels, choose a 64x64 column region.
Q
When making predictions how do I know how far ahead in timesteps is the prediction? Also can I calculate the probability of the prediction based on past events?
A
you always step ahead in time 1 frame at a time, so you always know how far ahead the prediction is. As far as detecting anomolies goes, it just depends how much the data differed from its predictions.
Q
How do I convert text or images to input that makes sense for region? With images I mean not black and white dots but let's say gray or even colored. With text do I have to give input one letter at a time (because the input size is fixed)?
A
this really depends on what you are doing and what youd like it to do. Its advanced, ive only ever used images myself, not even audio.
Q
I didn't see anything written about naming patterns in the documentation. How do I know which set of columns correspond to a pattern I want to know about? Do I have to tell the region that current input contains patterns "A", "B" and "C", then later I tell ok now this input contains "B" and "C" and it compares with previous input to find the common columns so it can narrow down which columns correspond to which names? If that is true then how would I do this and if I have multiple regions in layers which region do I tell this and which region should I later ask about it? Also is it possible with boosting and that stuff that columns that previously corresponded to "A" now correspond to something else or "A" is now different columns?
A
once something is learnt, it takes ages for it to forget it, youd have to change input completely for it to forget itself entirely.
so pretty much, as soon as a symantic has formed and its been there a while, it wasnt by accident, and its supposed to be there. as soon as a symantic forms, it also will borrow a lot of data to make it happen again, you barely ever would get a symantic changing locations or doubling.
I guess the only real way to know what a symantic is would be to poke that cell and see if something comes out the output after feeding back from it.
Last edit: Magnus Wootton 2013-05-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I've recently found out about HTMs and they seem very interesting. I'm just an amateur programmer and I have no knowledge of statistical models but I'm determined to understand how this theory works :)
I've read the Numenta documentation and looked at your project and I started to write my own implementation of it to better understand this. I have some questions I need help with.
When initializing region we create proximal synapses to a random set of inputs for each column. Is it possible that with this randomness some input bits don't have any synapses connected to them and does it matter if so?
Numenta documentation says each column has a natural center over the input and synapses in the center have a higher permanence when initializing but it does not say by how much, why is it necessary and how I should calculate this?
When permanence of distal synapses falls to zero should we delete them? I don't see anything mentioned about this in the documentation.
Numenta documentation talks about Xth input bit but then it also talks about center of input. Is the input supposed to be a matrix or just a series of bits and does it matter which?
How big should a region be compared to input and how many regions are needed? I know that larger regions means more complex patterns and more regions means almost the same but is a region supposed to be larger than input or smaller and how many regions is needed to solve some usual problems?
When making predictions how do I know how far ahead in timesteps is the prediction? Also can I calculate the probability of the prediction based on past events?
How do I convert text or images to input that makes sense for region? With images I mean not black and white dots but let's say gray or even colored. With text do I have to give input one letter at a time (because the input size is fixed)?
I didn't see anything written about naming patterns in the documentation. How do I know which set of columns correspond to a pattern I want to know about? Do I have to tell the region that current input contains patterns "A", "B" and "C", then later I tell ok now this input contains "B" and "C" and it compares with previous input to find the common columns so it can narrow down which columns correspond to which names? If that is true then how would I do this and if I have multiple regions in layers which region do I tell this and which region should I later ask about it? Also is it possible with boosting and that stuff that columns that previously corresponded to "A" now correspond to something else or "A" is now different columns?
Is there a more thorough documentation than Numenta's on HTMs that I could read?
Thank you
Hi,
I'm not a big expert on HTMs yet, so you better wait until our more experienced developers notice this post.
Look here for more links
https://sourceforge.net/p/openhtm/discussion/general/thread/ee9e1623/
Q
When initializing region we create proximal synapses to a random set of inputs for each column. Is it possible that with this randomness some input bits don't have any synapses connected to them and does it matter if so?
A
doesnt matter, if the synapse completely disconnects, you rerandomize it to read the current pattern coming in.
Q
Numenta documentation says each column has a natural center over the input and synapses in the center have a higher permanence when initializing but it does not say by how much, why is it necessary and how I should calculate this?
A
I say just randomize it completely, then youll be sharing the space completely between symantics, and they all produce bit clouds, in the right combination they should make an image. you will probably find with pure random youll be making distinct sizes and shapes anyway.
Q
When permanence of distal synapses falls to zero should we delete them? I don't see anything mentioned about this in the documentation.
Numenta documentation talks about Xth input bit but then it also talks about center of input. Is the input supposed to be a matrix or just a series of bits and does it matter which?
A
when a distal synapse falls to zero you rerandomize its location to be a part of the entering data pattern, so its always got a chance of coming back to reuse.
Q
How big should a region be compared to input and how many regions are needed? I know that larger regions means more complex patterns and more regions means almost the same but is a region supposed to be larger than input or smaller and how many regions is needed to solve some usual problems?
A
I say equal size to data complexity, if its 64x64 1 bit pixels, choose a 64x64 column region.
Q
When making predictions how do I know how far ahead in timesteps is the prediction? Also can I calculate the probability of the prediction based on past events?
A
you always step ahead in time 1 frame at a time, so you always know how far ahead the prediction is. As far as detecting anomolies goes, it just depends how much the data differed from its predictions.
Q
How do I convert text or images to input that makes sense for region? With images I mean not black and white dots but let's say gray or even colored. With text do I have to give input one letter at a time (because the input size is fixed)?
A
this really depends on what you are doing and what youd like it to do. Its advanced, ive only ever used images myself, not even audio.
Q
I didn't see anything written about naming patterns in the documentation. How do I know which set of columns correspond to a pattern I want to know about? Do I have to tell the region that current input contains patterns "A", "B" and "C", then later I tell ok now this input contains "B" and "C" and it compares with previous input to find the common columns so it can narrow down which columns correspond to which names? If that is true then how would I do this and if I have multiple regions in layers which region do I tell this and which region should I later ask about it? Also is it possible with boosting and that stuff that columns that previously corresponded to "A" now correspond to something else or "A" is now different columns?
A
once something is learnt, it takes ages for it to forget it, youd have to change input completely for it to forget itself entirely.
so pretty much, as soon as a symantic has formed and its been there a while, it wasnt by accident, and its supposed to be there. as soon as a symantic forms, it also will borrow a lot of data to make it happen again, you barely ever would get a symantic changing locations or doubling.
I guess the only real way to know what a symantic is would be to poke that cell and see if something comes out the output after feeding back from it.
Last edit: Magnus Wootton 2013-05-17