(Hi guys, this is interesting discussion that was started by Jojon in the old Numenta forum. I'd interesting if we deep into on logic reasoning. Is the logic reasoning intrisically embedded on HTM or not? What are your thoughts?)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
scottr wrote:
The temporal pooler is recursive in a way that might be turing complete, so it may be capable of any thought process, especially if it keeps sequential knowledge. I think it just depends on the HTM implementation and how you interpret the results. If it is only trained on 1->6 and the net of almost any type has 7 binary outputs that are restricted to having 1 active at a time, then 1 is not trained and it will always be the most likely on any unknwon input, giving the correct answer. But he may have a different concept in mind when he says A: he might mean it as a variable instead of a symbol like "6". So then it would require the HTM to know what he means by "A" in this question before it could give the correct answer.
It is not logical for a uneducatedperson (with no knowledge of what a number is) to know A+0=A given the above training, but he can likewise make the correct guess if he assumes A's are like the other symbols. He would also not have anymore knowledge about numbers, variables and addition than the simplest net.
Last edit: David Ragazzi 2013-02-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry, I should not have limited the numbers from 1 to 6. I should have appended "etc."
scottr wrote:But he may have a different concept in mind when he says A: he might mean it as a variable instead of a symbol like "6".
Yes, that is the direction I am going to: "A" as a variable. But you are one step ahead already. I didn't say a variable because I was thinking arithmetic and not yet algebra. For example, concerning arithmetic, I want to know if HTM can learn the base-10 number system.
Will HTM be able to predict that the subsequent symbols are “60 + 1”?
Nevertheless, let's use the term "variable" since it will greatly clarify the subject matter, and that's the direction I am going to anyway (after all, a child should learn algebra after arithmetic). Thanks for your keen observation.
scottr wrote:he can likewise make the correct guess if he assumes A's are like the other symbols.
That's exactly what I wish, namely: HTM can make such a correct guess like he (a human or a child).
scottr wrote:He would also not have anymore knowledge about numbers, variables and addition than the simplest net.
Yes. If he only learns arithmetic symbols, he will only have the knowledge of arithmetic.
But, if he learns other symbols from another domain (in formal languages and/or in natural languages), he will also have the knowledge of that domain, e.g. finance.
In fact, I have built a model to simulate "he", i.e. a system which learns both formal languages and natural languages, based on a network which can predict the sequential patterns of symbols. This is part of my long term plan to build a financial application which can predict more accurately the value of an investment product (stock, bond, commodity, etc.) based on an enormous amount of historical structured data as well as unstructured data such as news of consumer trends, political events or Fed's statements.
Last edit: David Ragazzi 2013-02-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TaleStuff wrote:it would require reasoning which isn't an attribute of HTM.
I have thought the other way around, namely: If HTM can perform the symbol prediction like "A+0=A", then HTM has an attribute required for reasoning. For instance, let's take a famous logical reasoning (called syllogism) as an example.
Suppose the HTM is trained with the following data:
An animal can move. A dog is an animal. So, a dog can move.
An animal can move. A snake is an animal. So, a snake can move.
An animal can move. A fish is an animal. So, a fish can move.
If we input the following data:
An animal can move. A butterfly is an animal.
Will HTM be able to predict that the subsequent symbols are “So, a butterfly can move.”?
If the answer is Yes, then HTM can also learn any logic.
Note that:
1) One word (and not one letter) is to enter the HTM a time. If we make a region of 1,000 column and set 20 columns active, then every word will be represented by the activity of 20 columns. So, one word is considered as one symbol.
2) The training data can be simplified by removing the premise "An animal can move." I intentionally include the premise in every line in order to conform with the complete format of syllogism.
TaleStuff wrote:It's the missing link for intelligent machines.
I make a tentative conclusion that, if HTM has the capability to perform such logic as above, HTM will provide the link missing for intelligent machines.
Last edit: David Ragazzi 2013-02-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
scottr wrote:
An HTM can recognize a size, position, and orientation that it has not seen before. Is that not deductive?
I think the "missing link" in AI is only that computers do not process information in the same way as brains, or (in systems that adequately model neurons) it is only a question of scale. Leaving quantum computers out of the discussion, brains are probably capable of performing any algorithm, but like computers, its design makes it better at certain types of problems. Brains are designed for survival, learning, and solving problems with minimal amount of energy use. Computers are designed to do what they are told. We pay more respect to things like us, and demean things unlike us. We even use factually incorrect labels like "white" and "black" when we want to make a thing like us to seem unlike us. When we eat other mammals, we do not normally use the mammal's real name like cow and pig, but have no qualms about chicken. So we should expect even scientists immersed in A.I. to be underestimating how far computers have come. The more people learn, the more they learn we are very small in the universe. In the past this awakening referred only to our physical size and place relative to the Earth, then Sun, then Universe, then universes. Our mental place in the world that immediately surrounds us may not be much greater. As a less esoteric example and getting back to computers, a relational database is a heirarchical thinking machine and has tons of data entry people so that a few manager-level people can ask it questions and get answers. It replaces middle management. At some point, it will not be possible to distinguish Google from what people used to call a God who answers prayers for information, or match.com for those praying for love, or amazon for those praying for some item, or facebook for friends, etc. To what extent is the internet already a God with an intelligence that exceeds many people? You may say the internet is only the product of people, but that is like saying chemical reactions that create people are intelligent and people are without any real intelligence.
Getting back to what I really wanted to say: brains are a restricted type of computer. They are restricted in a way that allows them to reproduce and therefore solve common problems. Computers are more general than brains in some sense (if not in a turing sense). So computers might be too smart. I've known smart people who remember everything and are not very good at solving puzzles. Our problem is how to carve a problem solver out of something that has too many other thinking skills. It's not trying to find the missing like as much as it is to remove the excessive potential intelligence comouters have that is not like us.
Pharmaceutical companies have faced a similar problem. The body itself is a vast thinking machine in the same way the Earth was a thinking machine in "Hitchhikers Guide to the Galaxy". The body only "respects" chemicals like itself which are DNA-generated molecules made at low temperature and low pressure. But chemists approach finding chemical solutions with their full array of skills which usually means higher temp and pressure and other conditions not found in DNA-based nature, ending up in things the body does not "respect" and has toxicity because the body did not evolve to handle non-DNA based molecules. Pharmaceuticals have a shocking array of toxicities and negative interactions with each other that are not found in natural compounds. Getting back to the analogy: computer scientists may have too many non-human A.I. skills to properly model the brain so that we may not properly respect the results. So numenta's approach is ideal for those who want to relate to the results. We already have plenty of artifical intelligence. We want human intelligence so that we marvel at the results. Our economic machine also wants the same thing so we can replace expensive human thinkers in the same way we replaced human physical workers (which some say caused the great depression via the production line before we were able to create the welfare state). Construction did not experience great productivity gains, so letting a construction bubble form was a good way of transferring money to people out of manufacturing productivity gains and outsourcing that was requiring fewer and fewer US workers. So a temporary depression could be the result of increasing A.I. because sudden increases in productivity leaves fewer buyers of the excess output. The standard of living in the US has not improved in the past 30 years despite high technology and the rise of computers because we shifted resources to a useless financial system that encouraged construction along with large tax breaks to encourage both finance and excessive residential construction (4 times more sq ft per person than in 1970) away from the high productivity of useful manufacturing. It is possible that the rise of the AI machines is the reason behind these bad economic decisions.
A.I., like the bomb, is just a tool. A very high level of intelligence and cooperation among people is required to use these tools to improve human life rather than letting the tools evolve faster than people can keep up. The end result will be to use computation to most efficiently use energy to move matter in order to create more computing devices. Our economic machine is working furiously to take expensive people out of this picture.
Last edit: David Ragazzi 2013-02-18
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
scottr wrote:
It is the programmer's job to replace expensive human thinkers with machine thinkers. The world is quickly being divided up between those who program (the high paying jobs) and those who don't. Numenta and others conscious or unconscious goal is to ultimately replace programmers. It is up to voters to keep ahead of everyone's economic motives that have lead and will increasingly lead to unfortunate and unconscious consequences. We have failed for the past 30 years to improve our standard of living despite the increases in productivity and technology and inexpensive Asian labor. Despite the flood of stimulus (debt) that we've been allowed to accumulate and exchange for cheap labor thanks to the dollar being the world's reserve currency. Despite English being the world's business language. It's not an impossible stretch to say computers were a cause of the lack of progress. Granted, computers have made things a lot more interesting for me personally and probably everyone reading this, it's just that as far as the government CPI and other measures are concerned, the median lifestyle as measured by wages and constant dollars has not improved. If A.I. really succeeds in programming itself, there is not much economic need for people to help the machines propagate themselves. Solar cells are about 150 times more efficient than plants at converting sunlight to useable energy, and humans are about 30 times less efficient over the course of 24 hours than electrical motors. Moving an electron in a wire uses about 200,000 times less energy than neuron impulses. We can't compete with machines in physical or mental labor in an economic system. We are low temperature and low pressure self-replicating machines who have managed to create high temperature, high pressure machines and are getting close to helping them build themselves. We are working very hard to help the machines self-organize to use their potential, replacing other human workers as fast as we can. This is an evolutionarily unavoidable problem (for people) that voters and governments are not equipped to foresee or deal with. Bill Joy is an Electrical Engineer who co-founded Sun Microsystems and wrote a famous paper in 2000http://www.wired.com/wired/archive/8.04/joy.html expressing the same concerns (in response to Ray Kurzweil's optimism). I am an electrical engineer who likes to work on problems that help people without regard to money, but I see no choice but to go along with the current program that Numenta and others are following. It's not merely economic motivation that will cause us to work in order to make people obsolete, but economics will make it happen rapidly. Dileep says 15 years before this technology is bigger than the internet. My guess is 25 to 50 years before it is bigger than people.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TaleStuff wrote:HTM in its present state isn't capable of deductive logic, it can only attempt to predict things that it has memorized.
I have a different point of view:
1) HTM attempts to predict not only things that it has memorized but also their sequential patterns.
Deductive logic is nothing more than sequential patterns plus instantiation (or analogy in general).
2) Although HTM isn't designed to perform deductive logic, it may be capable of deductive logic.
The animal brain was "designed" only for perception and behaviour, and it happens that in human the same brain's structure and algorithms can be used for deductive logic. The following is a quote from "The Tell-Tale Brain" by V.S. Ramachandran: "... many of our unique mental traits seem to have evolved through the novel deployment of brain structures that originally evolved for other reasons. This happens all the time in evolution. Feathers evolved from scales whose original role was insulation rather than flight. The wings of bats and pterodactyls are modifications of forelimbs originally designed for walking. Our lungs developed from the swim bladders of fish which evolved for buoyancy control. The opportunistic, “happenstantial” nature of evolution has been championed by many authors, most notably Stephen Jay Gould in his famous essays on natural history. I argue that the same principle applies with even greater force to the evolution of the human brain. Evolution found ways to radically repurpose many functions of the ape brain to create entirely new functions."
If HTM faithfully simulates the brain's algorithms for perception (in the occipital lobe), then HTM should be capable of deductive logic (in the frontal lobe) because the brain structures in both lobes are identical.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
scottr wrote:An HTM can recognize a size, position, and orientation that it has not seen before. Is that not deductive?
I would say that is inductive, because inductive is drawing a conclusion from particular to general, whereas deductive is from general to particular. Nevertheless, both are logic.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
teddybot wrote:
How are invariant representations for quantities processed?
When you think of '232' what are you seeing?
Most of us can not say that there are 62 pennies at a glance while a savant may. Some savants spent time counting items in their environment. Perhaps their mind is making invariant representations for quantities which are tied to levels of sparsity.
Daniel Tammet hears a number and sees a specific shape. http://www.youtube.com/watch?v=qXG-1YLGAS0
If asked to do an operation he sees a shape emerge between two others which is the answer to a hard math problem.
Are the shapes being used in the calculation or are the shapes merely what the brain is making of the underlying calculations?
Does this prove a superior neuronal math exists?
Lets say we show the HTM a 5 in binary and also set another cell on which means addition. Then we show it a 10. Next a 15.
It does learn a multidimensional relation between the binary in 5 and 10 associating to 15 but is that enough to get it to learn the concept of 'one more'
Given enough numbers it may figure out otherwise unknown associations between bits which gives mathematical shortcuts for answers based on rules governing their binary factors.
The problem with the chimp is that he NEVER gets the concept of ONE MORE.
Teaching him to count to 8 then 9 takes equal lessons as training him to count from 3 to 4. This is interesting because they do show concern for unfair quantities of food passed out. They are aware of relative amounts but not consciously a growing chain or sequence apparently- they need to add some circuitry in and around their HTM- perhaps some sort of feedback loop is missing which processes abstraction one level higher. "Each time we do this we go one farther than the last time"
What circuitry?
On Intelligence and searches on 'cortical loops' helps.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
teddybot wrote:How are invariant representations for quantities processed?
When you think of '232' what are you seeing?
Most of us can not say that there are 62 pennies at a glance while a savant may. Some savants spent time counting items in their environment. Perhaps their mind is making invariant representations for quantities which are tied to levels of sparsity.
Daniel Tammet hears a number and sees a specific shape. http://www.youtube.com/watch?v=qXG-1YLGAS0
If asked to do an operation he sees a shape emerge between two others which is the answer to a hard math problem.
Are the shapes being used in the calculation or are the shapes merely what the brain is making of the underlying calculations?
Does this prove a superior neuronal math exists?
Lets say we show the HTM a 5 in binary and also set another cell on which means addition. Then we show it a 10. Next a 15.
It does learn a multidimensional relation between the binary in 5 and 10 associating to 15 but is that enough to get it to learn the concept of 'one more'
Given enough numbers it may figure out otherwise unknown associations between bits which gives mathematical shortcuts for answers based on rules governing their binary factors.
The problem with the chimp is that he NEVER gets the concept of ONE MORE.
Teaching him to count to 8 then 9 takes equal lessons as training him to count from 3 to 4. This is interesting because they do show concern for unfair quantities of food passed out. They are aware of relative amounts but not consciously a growing chain or sequence apparently- they need to add some circuitry in and around their HTM- perhaps some sort of feedback loop is missing which processes abstraction one level higher. "Each time we do this we go one farther than the last time"
What circuitry?
On Intelligence and searches on 'cortical loops' helps.
So,if I train the network in this way:
input images{one apple,two apple.........one orange,two orange........one banana,two banana......}
output{1,2.........1,2..........1,2}
Then when I give some pictures with single or several objects(one picture)
will the network give the number of the objects in the picture?
If it gives, is there any possibility to train math operators based on this?
If it works then simple logic is achieved.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
teddybot wrote:
I don't think so.
This is where eye saccades, motor output and higher level circuits that 'count' may come in.
What we have with HTM is a bottom up pattern recognizer.
If 5 things are shown in two pictures in different positions not having eye saccades, motor output and higher level circuit will cause you to have to move it around and around but still it won't 'get it' because higher level circuitry is left out of the design.
I am working on an HTM toolkit so you can try it with clicking in your browser and visually see what needs to be done to make it work. Nobody on this site has agreed with me about this concept of development so it is going slow for me to develop and I see a bunch of words in posts and no code experiments at the forum.
What you want to try is easy to do with an on screen HTM toolkit which does not exist.
To get it to work at best the operand may need to take more inputs so the system is able to choose a category based on the operand. Without extra circuitry to give certain nodes more power or less power dynamically we just have a bottom up pattern categorize.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TaleStuff wrote:
Well, I agree with what you have said here Teddybot :-)
Too many people think that the current HTM technology has the capabilities of intelligence that Jeff described in the book while in fact it is just the method of storing a representation of a piece of its environment and recognizing when it occurs again. Without support circuitry (yet to be invented) including 'goal seeking' it won't become a 'logical thinking machine'.
scottr wrote:
I agree that it seems to have a lot missing by not having feedback down from the higher levels, but I am not sure anymore that this is a requirement. I do not think being able to affect the outside world is necessary for being and acquiring extreme intelligence. I also previously said it was missing "desire". But while converting the language into an economics, I realized the learning algorithm has the desire of becoming a predictor, changing itself in response to the environment which some would call a life with intelligence, eventhough it does not affect the world outside of itself, other than what people do in response to watching it. It uses people to reproduce ;) In inference mode, it is only a predictor.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TaleStuff wrote:
I guess it all depends on where we draw the line defining intelligence & life-form, an old mercury thermometer changes its physical appearance in response to the environment ;-)
scottr wrote:
But it doesn't go through a logical sequence that seeks an increase in its ability to predict. Conversely, we're just algorithms too, frequently changing out the atoms we think of as "us". We're ghosts more than machines.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TaleStuff wrote:
This is getting pretty weird;-) An Intel 4004 microprocessor could be programmed to increase it's ability to perform a task. Thing is, our brain's algorithms have the right stuff.
I've written some programs that learn but they aren't alive (well, maybe M5 is ;)
Idea for a math test:
HTM has a 3 bit input (no sparsification) and training assumes addition.
Possible math problems 2^3:
1) 0+0+0
2) 0+0+1
...
8) 1+1+1
Possible answers:
0,1,2,3
Your question is if the network is trained on 3 of the 4 answers above will it learn to get the 4th on its own?
Given the bitwise rules to add n numbers together out of a total possible m addition problems how many of m-n will come up with the correct answer?
Is the accuracy dependent on which set of n the network is exposed to possible to bit overlap?
If not all answers are right why are some bits wrong?
Can bit flipping code be added to get them right while training?
Is it possible to get it to work without extra circuitry around it?
If not what is required around it to get it to work?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
TaleStuff wrote:Too many people think that the current HTM technology has the capabilities of intelligence that Jeff described in the book
Some people believe it, and some people don't. Those who believe it get the benefit of being inspired. I like to believe it because I like to get inspired :)
TaleStuff wrote:Without support circuitry (yet to be invented) including 'goal seeking' it won't become a 'logical thinking machine'.
You're right. That's what I was thinking. in fact, theoritically I have figured out how a circuitry of HTM networks can become a logical thinking machine. I will need 3 HTM networks:
1) To learn the pattern of an equation (in math) or an argument (in logic).
2) To learn the pattern of instantiation.
3) To function as a controller, which coordinates the other 2 networks. Doesn't the prefrontal cortex have an executive function like a controller?
Needless to say that, the 3 networks are not to be connected in a hierarchical structure.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
teddybot wrote:Idea for a math test:
HTM has a 3 bit input (no sparsification) and training assumes addition.
Possible math problems 2^3:
1) 0+0+0
2) 0+0+1
...
8) 1+1+1
Possible answers:
0,1,2,3
Your question is if the network is trained on 3 of the 4 answers above will it learn to get the 4th on its own?
Given the bitwise rules to add n numbers together out of a total possible m addition problems how many of m-n will come up with the correct answer?
Is the accuracy dependent on which set of n the network is exposed to possible to bit overlap?
If not all answers are right why are some bits wrong?
Can bit flipping code be added to get them right while training?
Is it possible to get it to work without extra circuitry around it?
If not what is required around it to get it to work?
This is a math test. However, this test doesn't rely on a sequential pattern of symbols. Even if HTM can pass this test, it still cannot perform logic, e.g. syllogism as described in my previous post (page 1), because logic is inherently a sequential pattern of symbols.
My original question came from a top-down approach of a system design. I started from the desired intelligence machine and treated a pattern recognition system like HTM as a component. More specifically, I have designed an intelligence machine with the following steps:
1) First of all, I defined an intelligence machine as a system which can learn logic (formal language) and semantic (natural language). It may not have its own desire, motivation or goal. The goal may come from a human.
One application of the intelligence machine is financial analysis. One goal of the application is to predict the value of an investment product (stock, bond, gold, real estate, etc). The other goal of the application is to manage an investment portfolio according the predefined financial goals.
Different people have different definitions of intelligence, but that's my definition. The rationale is: If a machine can learn logic and semantic, it can match a human intellectual intelligence. The test is not a Turing test. The test is whether the intelligence machine can be educated at a university level and earn a university degree. In case of the financial application, the test is whether the intelligence machine can learn the CFA (Chartered Financial Analyst) materials, and pass all 3 levels of CFA exams.
Note that, the machine can also be creative as long as it is trained to be creative. The truth of this statement depends on the definition of creativity. In my case, I have taken the concept of creativity from Douglas Hofstadter as explained in the chapter "Variation of a theme as a crux of creativity" of "Metamagical Themas". This is consistent with a research by Mihaly Csikszentmihalyi, which is documented in his book "Creativity: Flow and the Psychology of Discovery and Invention".
2) After years of research on how a human learns English and math, and borrowing concepts from AI and simulation experts (primarily Steven Pinker, Marvin Minsky and Grady Booch), I came up with the components necessary to build the intelligence machine. It turned out that one critical component is a sequential symbol prediction engine.
3) Now, I have to find a technology which can predict the sequential pattern of symbols. I would like to find out if HTM can function like my fictitious sequential symbol prediction engine. It seems that a circuitry of HTM networks will do.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
(Hi guys, this is interesting discussion that was started by Jojon in the old Numenta forum. I'd interesting if we deep into on logic reasoning. Is the logic reasoning intrisically embedded on HTM or not? What are your thoughts?)
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18
Last edit: David Ragazzi 2013-02-18