Chomsky Vs Norvig And The Missing Debate About The Nature Of Intelligence

[Kriminal99]

Understanding is not irrelevant in Norvig's approach because of the problem of overfit to data. Without an accurate means of generalizing from patterns in the data, you are limited in how you can apply that knowledge to data outside of the range.

 

You could build an enormous database of knowledge and just look everything up, but an AI with a human like ability to generalize wouldn't need anywhere near the same amount of storage space or processing power. Such a program would use methods of generalization specifically because they were successful in associating the most amount of information, and could do it without storing every little piece of information.

 

Example: You ask a person to classify photos of rooms as classrooms or non classrooms.

 

A program that looks for chalkboards at the front of the room misses ones that have white boards, smart boards, or projector screens.

 

A program that tries to look for any member of the set of all of these things takes a lot of work to put in this a priori knowledge, and requires constant updates because someone forgot a new communication medium or a new one was invented.

 

A human like program would generate a concept of "communication medium", not just by using a set of all known examples but by another process and would need much less stored information to achieve the same effect. This generation is automatic after following some relatively simple hard programming.

 

Mapping this program to a serial computer would require some corner cutting to bypass the "content addressable memory" properties of physical neural nets and the computational complexity they save.

 

I guess my point is, in the end all 3 of these methods get to the same place when you just add enough information and understanding to the system to the point where nobody can tell the difference which one. But a real strong AI would be so much more flexible, generate it's own knowledge, and require less maintenance

[Doctordick]

So, Norvig, meanwhile, represents the new philosophy: truth by statistics! Anyone who manages to comprehend the essence of my presentation would also comprehend that it is a logical defense of Norvig's position (the only truly rational position). :P

 

The crux of this debate and analysis of "who's winning" has really come down to the apparent success of the Neural Network/Statistical Learning approaches versus the general lack thereof from grammar-based approaches, which have been in development for a lot longer.

Buffy, you are absolutely correct: “success” is the critical measure. The real problem is that “the problem” is actually unbelievably complex; “intellegence” does not solve the problem at all, it only approximates a solution (essentially that is better than nothing).

 

When I think about how to translate these two points of view, I see a huge difference in the two combatants goals:

Yes, Chomsky's goal is based upon the idea he can understand reality! Norvig's goal is based upon the idea that that the expectation of “future circumstances” can be statistically calculated as consequences of the present circumstances. If the explanation of reality is “What is” is “what is” and there is nothing to understand, Chomsky is wrong and Norvig is right! I have proved that “What is” is “what is” is the only correct explanation of reality. Life is “success”: a collection of circumstances (macroscopic collections of events) which provide valid statistics for the behavior which yields survival.

 

As I have said, explanations are procedures (think computer programs) which produce survival enhancing behavior. Understanding is an unnecessary figment of one's imagination.

 

My interpretation of what you're trying to get at with bottom middle and top is this:

Bottom = sensory input

Middle = neural net that processes the input

Top = conceptual understanding of the meaning of the inputs

 

 

If that's what you mean, then I think I agree: that in fact maps onto how the brain actually developed over the last 250 million years.

Your understanding of “how the brain actually developed over the last 250 million years.” My understanding is simply that it came to be because of its success. Note your assumption of “sensory input”; explain how the brain came to “comprehend that”. ;)

 

our intelligence is rooted not in what we answer, but in what we ask.

I would go along with that!

 

My feeling is that human intelligence is a hodgepodge of data-processing stratagem aimed at survival/procreation in the peculiar, intensely social & intellectually competitive environment that humans find themselves evolving in.

...

An engineered intelligence, not encumbered with ego or zeal, should immediately see how these two philosophies need one another more than they need to argue.

A man after my own heart! However, pure “all encompassing” statistics is an unbelievably complex solution. A few short cuts (modern science) make things a lot easier. The issue I have proved in my work is that modern science is no more than solution aspects of that “all encompassing” statistical attack. At least I prove that modern science is a Norvig solution. An issue which seems to be above the comprehension of most here. To quote Turtle, what we ask is the critical issue; no one seems to even think about the question I am asking in my presentation: "what follows from definition and nothing else?" :o

 

This issue became clear as it was discovered that the parallel postulate was not necessarily valid and its applicability was an empirical matter, deciding whether the applicable geometry was Euclidean or non-Euclidean.

What everyone tends to omit here is that, if the coordinates of the geometry are to be totally independent of one another (including scale), the parallel postulate is absolutely necessary. Anytime a representation deviates from Euclidean geometry, an assumption upon the relationship between coordinates has been made.

 

Just a quick comment!

 

Relying on words to lead you to the truth is like relying on an incomplete formal system to lead you to the truth. A formal system will give you some truths, but as we shall soon see, a formal system, no matter how powerful—cannot lead to all truths,

Ah, and you think you can prove that do you?

 

Life is made up of constant calls to action, and we seldom have time for more than hastily contrived answers,

Ah, people just want “simple” explanations. But the truth isn't simple unless you see it as “What is” is “what is”. I always found it funny that God told Moses his name was “I am that I am”. Maybe he knew what I know.

 

You're bit about the Norvig / Chomsky debate reminded me of a quote by Ashleigh Brilliant 'Life is the only game, who sole purpose is to discover the rules of the game.'

Has it ever occurred to you that there might not be any rules? That would make Norvig correct! Plus that, I hope you have noticed that the breakthroughs in science are never made by the “the old man who has crossed this sea a thousand times”!

 

Without an accurate means of generalizing from patterns in the data, you are limited in how you can apply that knowledge to data outside of the range.

...

But a real strong AI would be so much more flexible, generate it's own knowledge, and require less maintenance

To quote myself above, “Pure 'all encompassing' statistics is an unbelievably complex solution. A few short cuts (modern science) make things a lot easier. “

 

Personally I think that is why our brains are divided up into different compartments. In the deep and dark past, some very important operations turned out to be central to survival and those “means of generalizing from patterns in the data” were retained. Of course, that's only an opinion and it is clearly based upon the assumption that “the structure of our brain” is a necessary part of our concept of reality.

 

The problem is, finding those "means of generalizing" by accident, takes a long time. :D

 

As a conscious being I am involved in a story. The perceiving part of my mind tells me a story of a world around me. The story tells of familiar objects. It tells of colours, sounds, scents belonging to these objects; of boundless space in which they have their existence, and of an ever-rolling stream of time bringing change and incident. It tells of other life than mine busy about its own purposes. As a scientist I have become mistrustful of this story.

 

Sir Arthur Eddington, 1934

from his book: “New Pathways in Science”

Rade confounds this with something Eddington said some six years earlier. If one reads “New Pathways in Science” further, the “New” being significant there, Eddington essentially finds the problem impossible to solve and decides it is not solvable. He essentially misses the fact that “generating a language” is part of the solution and can not be presumed given.

 

 

Have fun – Dick

 

Tis sometimes the height of wisdom to feign stupidity, :phones:

Buffy

Especially when the population becomes large. Stupidity often out survives conventional intelligence. Even the Romans had problems it getting their upper classes to reproduce and look at Japan today, one of the most intelligent collection of people out there are just not replacing themselves. :(

Edited March 23, 2013 by Doctordick

[JMJones0424]

Chomsky's assertions, especially regarding linguistics, have generally rubbed me the wrong way, but I lack the educational foundation to argue against prevailing opinion. A very interesting article came up on my RSS feed today regarding modern western psychology, especially in the US. Not only are current opinions swayed heavily by incomplete sampling (subjects of most studies on cognition tend to be college educated westerners), but spatial optical illusions tend to be heavily influenced by culture.

 

...

During our dinner, I admitted to Heine, Henrich, and Norenzayan that the idea that I can only perceive reality through a distorted cultural lens was unnerving. For me the notion raised all sorts of metaphysical questions: Is my thinking so strange that I have little hope of understanding people from other cultures? Can I mold my own psyche or the psyches of my children to be less WEIRD and more able to think like the rest of the world? If I did, would I be happier?

 

Henrich reacted with mild concern that I was taking this research so personally. He had not intended, he told me, for his work to be read as postmodern self-help advice. “I think we’re really interested in these questions for the questions’ sake,” he said.

 

The three insisted that their goal was not to say that one culturally shaped psychology was better or worse than another—only that we’ll never truly understand human behavior and cognition until we expand the sample pool beyond its current small slice of humanity. Despite these assurances, however, I found it hard not to read a message between the lines of their research. When they write, for example, that weird [western, especially US] children develop their understanding of the natural world in a “culturally and experientially impoverished environment” and that they are in this way the equivalent of “malnourished children,” it’s difficult to see this as a good thing.

...

 

I don't know why the idea that one perceives reality through one's cultural lens would be unnerving, to me it seems obvious. As language is a significant part of culture, and Chomsky's ideas on cognition seem to me to be centered around a universal, context-free grammar, I count myself happily in the Norvig camp.

[JMJones0424]

Norvig is looking for a way to gather together enough data, such that programs produce "correct results" but "understanding" is really irrelevant: no matter what you want the software to do, there's an analogue out there that's close enough that you'll get the right result a percentage of the time that is proportional to the amount of modelling data that you can get your hands on.

...

When I think about Norvig's argument I can't help but think of the old saw about if you have enough monkey's typing on enough typewriters, eventually one of them will type the entire works of Shakespeare, the problem being the definition of the word "enough". Norvig's logic really completely depends on--to quote Shakepeare--there being (almost) nothing new under the sun. Unless you have something "close" to a desired result in your learning set, your neural network is unlikely to produce that result.

 

I don't see it this way at all, though admittedly, I may be looking at it incorrectly. I don't think understanding is irrelevant, instead I think understanding, in the instance you are using it, is a myth.

[Buffy]

Wow! :cheer:

 

I am quite honestly and sincerely honored that you agree with me Doctor!

 

The beauty of your theory of course is that it comes "from nothing" and to misquote Hannibal Smith: I love it when a plan comes together from nothing!

 

The only quote of yours I want to pull here is, "...pure “all encompassing” statistics is an unbelievably complex solution. A few short cuts (modern science) make things a lot easier." Because that's exactly the point I was trying to make in creating this thread.

 

I happen to love that the real world works from those "all encompassing statistics", but I as a computer scientist like to mix and match to get the results as quickly as possible. I don't think either Chomsky or Norvig are "wrong": I think it's their fanboys who make absolutist "we're right and they're wrong" arguments that really form an impediment to "getting real work done."

 

When I'm debating abiogenisis however, I happen to love pure statistical randomness and a Universal Representation of Rules!

 

I have been meaning to catch up on that thread, so maybe that'll be some heavy reading for tomorrow...

 

Thanks again! :cheer:

 

You can't learn too soon that the most useful thing about a principle is that it can always be sacrificed to expediency, :phones:

Buffy

[Buffy]

Chomsky's assertions, especially regarding linguistics, have generally rubbed me the wrong way, but I lack the educational foundation to argue against prevailing opinion.

...

I don't know why the idea that one perceives reality through one's cultural lens would be unnerving, to me it seems obvious. As language is a significant part of culture, and Chomsky's ideas on cognition seem to me to be centered around a universal, context-free grammar, I count myself happily in the Norvig camp.

 

Even those of us who find Chomsky incredibly insightful and useful have problems with his "anti-nurture" bent, but he actually claims not to have that bias, it just comes across that way.

 

But the nature of the debate really comes down to the fact that what Norvig's followers have done is no more than some amazingly accurate parlor tricks. Natural Language translation was always the bugaboo of AI based on Chomsky's approach, and once pure statistical manipulation of data came along using huge text databases, we've seen translation go from 50% right to 90% right. Indubitably a major step forward. But the thing that it lacks which is a side effect of Chomsky's approach is the fact that it spits out "semantics" not just "whenever someone says 'The sky is blue' in English, a Russian speaker would say '

Небо является синий'" If you want to *do* something with that natural language like respond to it, you need semantics or "undersanding". This is exactly the problem that the folks at IBM ran into with their Jeopardy playing computer and what caused it to spit out some amazingly humorous responses.

 

Now as Doctor Dick explains above, you don't strictly speaking *need* that set of logical/semantic axioms to get it to work, if you want to do it in less than a millenia, it might be useful to throw in some outside rules to follow as givens, and that's exactly what the folks at IBM ended up doing with Watson.

 

But that's just being *practical*! :cheer:

 

Would you like me to give you a formula for success? It's quite simple, really. Double your rate of failure, :phones:

Buffy

[Doctordick]

Wow! :cheer:

 

I am quite honestly and sincerely honored that you agree with me Doctor!

I have read a lot of your posts and very seldom have any disagreement with you. I had always hoped that my presentation would interest you more than it seems to have as I see you as quite intelligent.

 

I have only one complaint though!

 

The beauty of your theory of course is that it comes "from nothing" and to misquote Hannibal Smith: I love it when a plan comes together from nothing!

Everyone seems to think that I am presenting a theory! It is not a theory at all. It is no more than a deduction from my definitions. That is an important issue and creates unbelievable problems when people simply refuse to think in terms of my definitions.

 

I happen to love that the real world works from those "all encompassing statistics", but I as a computer scientist like to mix and match to get the results as quickly as possible. I don't think either Chomsky or Norvig are "wrong": I think it's their fanboys who make absolutist "we're right and they're wrong" arguments that really form an impediment to "getting real work done."

I don't disagree with that at all. I have tried to make that clear to Anssi and Bombadil (who seem to be the only people interested in understanding what I have to say) but have pretty well failed so far. I have never really managed what I would call a usable solution to my equation. What I have managed to do over the last fifty years is to show that many (if not all) of those theoretical representations of modern physics are approximations to that equation: i.e., I have found no evidence that science is not a tautology.

 

What I think I have shown is that Norvig is right. But that doesn't make his approach the best. In private messages, Anssi and I have been discussing the design of a computer program which would converge towards decent predictions of the future. That is not so difficult (I had written a simple version of such a thing maybe thirty years ago). The real problem in AI is to design a system which converges towards “proper actions”, essentially intellectual survival, a substantially different goal. However, that is the very essence of evolution and perhaps not what you really want anyway. As I said earlier, intelligence and survival are not exactly the same thing.

 

The problem is, finding those "means of generalizing" by accident, takes a long time. :D

 

I have been meaning to catch up on that thread, so maybe that'll be some heavy reading for tomorrow...

It might be better to start a new thread with your questions. You can always paste things over from that thread!

 

You can't learn too soon that the most useful thing about a principle is that it can always be sacrificed to expediency, :phones:

Buffy

Isn't that the central theme of life??

 

But that's just being *practical*! :cheer:

Luckily, I don't have to be *practical*; I am retired and have more than I really want to take care of. :lol:

 

Have fun -- Dick

[Turtle]

i did some sleuthing to put the quotes right. the system seems predisposed to mis-lose attributions.

 

 

...

As I have said, explanations are procedures (think computer programs) which produce survival enhancing behavior. Understanding is an unnecessary figment of one's imagination.

 

 

... hoovering up data, no matter how much time is allotted, and putting it in a pre-made container is building from the middle up. human intelligence, as well as human bodies, are made from the bottom up. ...

 

My interpretation of what you're trying to get at with bottom middle and top is this:

• Bottom = sensory input
• Middle = neural net that processes the input
• Top = conceptual understanding of the meaning of the inputs

...

 

 

Your understanding of “how the brain actually developed over the last 250 million years.” My understanding is simply that it came to be because of its success. Note your assumption of “sensory input”; explain how the brain came to “comprehend that”. ;)

 

 

so you see, with the quotes set right it is my "sensory input". just stuff going on "outside" of stuff going on "inside" and affecting the stuff inside. that you so quickly change terms -from "understanding" to "comprehend" for example- is vexing. moreover, you say "understanding" is a figment, and then present yours. :rolleyes:

 

...

our intelligence is rooted not in what we answer, but in what we ask. ...

 

I would go along with that!

 

sehr gut! :dogwalk:

 

 

...... To quote Turtle, what we ask is the critical issue; no one seems to even think about the question I am asking in my presentation: "what follows from definition and nothing else?" :o

 

definition, and nothing else.

 

...

...

The problem is, finding those "means of generalizing" by accident, takes a long time. :D

 

i think we have a whole thread here someplace all on what an "accident" is, but in any case taking a long time is not a problem. a long time has passed and every [statistical] indication is that there is yet a long time to go.

 

...

What other aspects of intelligence do you espy?

 

When you're not looking at it, this sentence is in Spanish, :phones:

Buffy

 

 

perhaps the nature of intelligence is simply "long time", in which case when you head out for AI, it may not matter that you take the Norvig superhighway or the Chomsky scenic route. :steering: you get tied up by traffic on the highway and slowed by terrain on the byway. if you have to ask "are we there yet?" , then we aren't. :lol:

Edited March 24, 2013 by Turtle

[Rade]

I was reading this article the other day and it got me thinking about a topic that's been touched on in the True Ai, Are We Getting Close Yet? thread: a lot of the heated rhetoric in the community concerning AI really has everything to do with a lack of a common understanding of the word "Intelligence", and maybe even the word "Artificial".

Hello. In the article link you provide is the comment below, which in summary suggests that BOTH Chomsky and Norvig are incorrect, and that the correct path is to discover how chaos can serve as a dialectic for a machine entity that will allow it to unite the linear determinism of Chomsky with the stochastic statistical approach of Norvig...do you find any utility in chaos for machine intelligence (or AI) given that a chaotic machine functions in a reality with complete lack of constraint...I mean, is this not what happens in the human mind to make us "intelligent" ?

 

Edit: See here for example the association of chaos and AI:

http://www.lycaeum.org/~sputnik/Misc/chaos.html

 

==

 

COMMENT OF 27. splencha...Wednesday July 25, 2012 06:25pm EDT:

 

It strikes me that language is both linear (Chomsky) and stochastic (Norvig) in a non-reducible way, much like other not-very-predictable/computable phenomena like weather, evolution, and music. Which means that it can be modeled realistically by various combinations of random and functionally generated numbers, and the models statistically compared for ability to successfully generate rules and exceptions that duplicate a specific language in hindsight. This process only generates language in a computing machine that allows the generated language to alter the rules whilst generating the particles from which the language is, itself, made. Neither Chomsky nor Norvig does this, though they both work very hard at what they do, with some results.

 

In short, I think they are both wrong, in ways that cannot be overcome by applying more effort to the processes that have gotten each of them as far as they have, thus far, come.

 

There is a third strategy that I have not heard, recently, in this debate, of modeling an entity that can intersperse catastrophic error with the opposing deterministic linear and stochastic models in such a way that it actually learns English... .

Edited March 24, 2013 by Rade

[Rade]

...definition, and nothing else.

OK, I agree that what follows from 'definition and nothing else', is definition, but that does not mean the process stops at definition and does not lead to something else..correct ? So, suppose we have five definitions A,B,C,D,E that are contextually valid and five definitions F,G,H,I,J that are contextually false and worthless. Would you not agree that for definitions F,G,H,I,J any presentation that follows from them is worthless use of language, so something else does follow, we call it a 'waste of time' ? Then, would it not be true that what follows from valid definitions A,B,C,D,E would be an integration of known relationships of the concepts from which the definitions were derived, thus what follows from valid definition is the validity of all conclusions, inferences, thought, and knowledge ?

 

The important issue is that not all definitions are valid, some are worthless because they either (1) do not specify known relationships among essential characteristics of the concepts defined, or (2) they contradict what is known about concepts by omission and/or evasion.

 

In summary, what follows from valid definitions is knowledge itself, what follows from worthless definition is definition, and nothing else. Please let me know what you do not agree with.

[Turtle]

OK, I agree that what follows from 'definition and nothing else', is definition, but that does not mean the process stops at definition and does not lead to something else..correct ? So, suppose we have five definitions A,B,C,D,E that are contextually valid and five definitions F,G,H,I,J that are contextually false and worthless. Would you not agree that for definitions F,G,H,I,J any presentation that follows from them is worthless use of language, so something else does follow, we call it a 'waste of time' ? Then, would it not be true that what follows from valid definitions A,B,C,D,E would be an integration of known relationships of the concepts from which the definitions were derived, thus what follows from valid definition is the validity of all conclusions, inferences, thought, and knowledge ?

 

The important issue is that not all definitions are valid, some are worthless because they either (1) do not specify known relationships among essential characteristics of the concepts defined, or (2) they contradict what is known about concepts by omission and/or evasion.

 

In summary, what follows from valid definitions is knowledge itself, what follows from worthless definition is definition, and nothing else. Please let me know what you do not agree with.

 

 

philosophy- a route of many roads leading from nowhere to nothing. ~ ambrose bierce

[Rade]

philosophy- a route of many roads leading from nowhere to nothing. ~ ambrose bierce

So, for Bierce, philosophy as a human activity is an example of a strange loop, as you presented in another thread:

 

===

A strange loop is a hierarchy of levels, each of which is linked to at least one other by some type of relationship. A strange loop hierarchy, however, is "tangled" (Hofstadter refers to this as a "heterarchy"), in that there is no well defined highest or lowest level; moving through the levels one eventually returns to the starting point, i.e., the original level. Examples of strange loops that Hofstadter offers include: many of the works of M. C. Escher, the information flow network between DNA and enzymes through protein synthesis and DNA replication, and self-referential Gödelian statements in formal systems.

 

In I Am a Strange Loop, Hofstadter defines strange loops as follows:

 

“ And yet when I say "strange loop", I have something else in mind — a less concrete, more elusive notion. What I mean by "strange loop" is — here goes a first stab, anyway — not a physical circuit but an abstract loop in which, in the series of stages that constitute the cycling-around, there is a shift from one level of abstraction (or structure) to another, which feels like an upwards movement in a hierarchy, and yet somehow the successive "upward" shifts turn out to give rise to a closed cycle. That is, despite one's sense of departing ever further from one's origin, one winds up, to one's shock, exactly where one had started out. In short, a strange loop is a paradoxical level-crossing feedback loop. (pp. 101-102) ...

===

 

My interest in philosophy is not the origin and end of many roads traveled, but what abstractions I encounter during the journey.

[Turtle]

So, for Bierce, philosophy as a human activity is an example of a strange loop, as you presented in another thread:

 

bierce had no inkling of hofstadter before he bit the dust, therefore he could not have made an example of it.

 

===

A strange loop is a hierarchy of levels, each of which is linked to at least one other by some type of relationship. A strange loop hierarchy, however, is "tangled" (Hofstadter refers to this as a "heterarchy"), in that there is no well defined highest or lowest level; snip...

 

 

as luck has it i have have hofstadter's meticulously indexed I Am A Strange Loop right in my hand, and the word "heterarchy" is not indexed. could you tell me exactly what chapter/page/paragraph it is on and per se in so i can laud the error over dougie?

[Rade]

bierce had no inkling of hofstadter before he bit the dust, therefore he could not have made an example of it.

OK, but would Hofstadter find Bierce comment about philosophy to be a strange loop ?

 

as luck has it i have have hofstadter's meticulously indexed I Am A Strange Loop right in my hand' date=' and the word "heterarchy" is not indexed. could you tell me exactly what chapter/page/paragraph it is on and per se in so i can laud the error over dougie?[/quote']Well, the best I can do is link you to the thread where you presented the the word heterarchy as shown above....see post #11 in the thread titled "the most critical question" in philosophy of science.

 

Perhaps we stray much too far from the OP topic to continue dialog about strange-loops and valid and non valid use of definition ?

[Turtle]

OK, but would Hofstadter find Bierce comment about philosophy to be a strange loop ?

 

well, i can't speak for hofstadter but speaking for hofstadter i'd say that he would say that bierce is a strange loop and his comment is biting.

 

Well, the best I can do is link you to the thread where you presented the the word heterarchy as shown above....see post #11 in the thread titled "the most critical question" in philosophy of science.

 

Perhaps we stray much too far from the OP topic to continue dialog about strange-loops and valid and non valid use of definition ?

 

 

mmm....but uh...erhm....you didn't actually give a link!!?? allow me, as context is everything. >> post #11

 

see, y'all talk a good game, but you never actually play the game. anyway, in that post i was quoting wikipedia on the use of the term "heterarchy" & either they are wrong or dougie didn't index the term. i have yet to find an online copy of I Am a Strange Loop, so any direct quoting of it I have to transcribe by hand. i'll keep an eye out next time i re-read the work.

 

i'll leave it to buffy to decide if i have strayed.

[Kriminal99]

 

 

To quote myself above, “Pure 'all encompassing' statistics is an unbelievably complex solution. A few short cuts (modern science) make things a lot easier. “

 

Personally I think that is why our brains are divided up into different compartments. In the deep and dark past, some very important operations turned out to be central to survival and those “means of generalizing from patterns in the data” were retained. Of course, that's only an opinion and it is clearly based upon the assumption that “the structure of our brain” is a necessary part of our concept of reality.

 

The problem is, finding those "means of generalizing" by accident, takes a long time. :D

 

 

The way to generalize patterns from the data is all the human brain is. The different sections of the brain are just the end result after being given certain inputs over multiple generations. It didn't take any time to find either. It's modeled after a simplistic proxy for survival that is very easy to mechanically realize.

 

What I am essentially saying is that the two approaches are really the same thing. The hierarchies Chomsky is trying to create can actually be generated using statistical methods.

[AnssiH]

Opinions?

 

DD recommended me to take a look at this thread, so I did, and since I've got plenty of opinions, I guess I could spare a few... :D

 

The OP was talking a lot about natural languages in the context of Chomsky and Norvig, so first things first. People tend to tacitly equate "understanding of human languages" with "intelligence", and that view can lead into some pretty serious lapses of judgment when it comes to the philosophy of intelligence - or to the desire of building an artificial general learning system, which is to say, a system capable of forming novel useful concepts and models based on information whose meaning is completely unknown. Just like we can.

 

I can't say Norvig has made that mistake because he doesn't seem to be interested of the problem of general intelligence at all, but I would say Chomsky has made many comments that implies he is thinking of this thing backwards, as in "language first" and intelligence as an extension of language. As if being able to generalize human languages means we have generalized intelligence. That is exactly the mistake being made when people think Turing test has got some practical relevance. To me, this seems like an extremely naive way to view the topic.

 

Let's just start by defining what do we mean by "intelligence". By being intelligent, we mean we can learn to understand things we have not been pre-programmed to understand. We do NOT mean we "know everything there is to know about the world". We don't even mean "we understand the fundamentals of reality". We mean we can build ways to view anything. We can build *different* valid ways to view anything - even the entire world. This is also called general semantics.

 

Let me view this in sense of "general learning". General learning of course means ability to learn about anything without a-priori knowledge of it. In practice this means the learning is based on being able to generate some kind of model of that something. And in practice, since there is no way for the system to check whether or not its model is actually correct apart from generating correct predictions, what we are really talking about is ability to create a model that is capable of predicting something. That is exactly the case with our world views, and generating correct predictions is what we mean when we say we "understand" something. Ironically, this inability to check the correctedness of our ideas is exactly what leads to "semantical" understanding; we understand everything in terms of context we defined also ourselves, and we can change the definitions of that context in many valid ways leading into different perspectives on the same thing, without being able to differentiate the correctedness of any view.

 

Anyhow, the world is full of examples where animals who are not able to converse with us, nevertheless are able to learn to cope with situations that they are not in any "innate" sense equipped to cope with. Animals living in savanna, who have never seen ice, nevertheless have the ability to learn to recognize "ice", learn to conceptualize that it is very slippery, and learn to avoid it. They learn to predict, that if they walk on that shiny thing, it would be very hard to move. Birds learn to use human inventions to their advantage (traffic lights, for instance). I think it is evident they are building a semantical world model, and I call that "intelligence".

 

And when an organism is capable of building a large enough world model, it will start to conceptualize other organisms, and it will conceptualize itself, and it will start to conceptualize what it thinks the other organisms are conceptualizing, and it will start to conceptualize means of communication with those other organisms. Also called "natural language". There are plenty of examples where animals exhibit a limited ability to communicate with us.

 

What I'm getting at is that there is no reason to assume that our ability to use human language is in some fundamental way different kind of intelligence than the intelligence being exhibited when an animal learns to conceptualize "ice". Natural languages are an extrapolation of the same exact general learning mechanisms. Language is just a result of general intelligence. Whether or not there exists hard-wired short-cuts in our brain that makes it easier for us to learn human languages, philosophically speaking there is absolutely nothing in our ability to understand language, that is not entirely available to a general learning mechanism, since what we are doing anyway is just modeling a communications system.

 

In practice that means we will be able to build general learning systems that start to exhibity clear signs of being able to conceptualize novel ideas far far far before we are having an english language conversation with that system. At the same time, just increasing the capacity of the system of building more extensive world models will inevitably also lead to its ability to model human languages. Then we will be conversing with it, and it will also, in fact, "understand" human language in exactly the same sense as you and I "understand" it; also called semantics.

 

It is pretty obvious that Turing test is absolutely useless experiment in recognizing "intelligence". Real people can fail individual Turing tests, and machines can pass it sometimes, so what? It is easy to show with different tests whether or not the system or the person is capable of predictive modeling of information whose meaning it doesn't know. Show me a general learning mechanism that can learn to navigate fluently through novel environments, even if we cut some of its limbs. And let's see a system that can learn to use the environment to its advantage. Those would be good indications towards the fact that the system is modeling its own world in valid manner. Basically this ability is also in itself "semantical" understanding of the world.

 

There seems to be a lot of people out there who have not grasped this fact, and are instead attempting to define general rules of human language without concerning themselves with the problem of general learning at all. And that gets us back to Chomsky. Basically, what he is doing with "universal grammar" is an attempt to generalize human language, which is not at all the same thing as generalizing intelligence. It just means he is generalizing something we have built as part of our world view. Something I find entirely uninteresting. Even if he succeeds, he has only created an analysis of our present time human language, not a system capable of general learning.

 

Now in his mind universal grammar represents some kind of innate human ability of language, but there even exists some excellent linguistic evidence towards the idea that the rules of our languages are arbitrary accidents, based on the world views that generated the languages. Do we "think the way we speak", or do we "speak the way we think"? Well, the thing is that our languages are part of our world views, and with different paradigms we build very different kinds of languages. The Pirahã language is the best example of this, throwing a serious wrench into Chomsky's theory. It doesn't seem to have infinite recursion to it (universality of recursion is one aspect of Universal Grammar). It doesn't have numbers, it doesn't have abstract colors. Basically their language features seem to boil down to the fact that their language is not influenced by other languages, and at the same time their cultural philosophy of the world basically says that anything that you have not witnessed yourself is irrelevant (so there's very little use for recursive features in communication);

http://www.edge.org/3rd_culture/everett07/everett07_index.html

 

This also means it is extremely difficult for us to learn their language, since it requires a very big paradigm shift in your personal philosophy of the world (in fact Daniel Everett himself turned from fundamentalist Christian missionary into an atheist as a result of learning the philosophy of Pirahãs, but that's another story). And it is equally difficult for them to learn our languages. It is hard to imagine of not having a good conception of clauses and infinite recursions.

 

Personally, since I view human languages as a result of semantical world view building and not the other way around, I would say in principle it would be possible to create far far more alien communication schemes than Pirahãs, and learn to understand them just like we understand our own language. There's absolutely no reason to view Chomsky's Universal Grammar as anything else but merely an analysis of the languages we just so happen to currently use, and nothing else. For Pirahãs his perspective is exactly analogous to an english speaking person who thinks we must have an innate ability to understand english, because "how would we otherwise understand our thoughts; they are after all in english!"

 

Now a lot of controversy has ensude by the apparent lack of recursion in Pirahã language, and Chomsky himself has said something wonderfully meaningless about this issue, merely revealing a weakness in his philosophy. In Chomsky's view, what his "universal grammar" is about is some innate abilities of human beings to understand grammar; he thinks of some "genes" that give us the ability to understand grammar. According to the Wikipedia article about Pirahã language; "Noam Chomsky has replied that he considers recursion to be an innate cognitive capacity that is available for use in language but that the capacity may or may not manifest itself in any one particular language."

 

So what is that supposed to even mean? "innate cognitive capacity?" Is it "innate cognitive capacity" if we have the general learning capacity that allows us to form the concept of recursion, among any other concepts? If so, don't we all trivially agree that we indeed do have the capacity of learning the idea of recursion, since anyone who can read this text has evidently learned that idea?

 

Or, is Chomsky rather saying that recursion is not something we need to learn through general conceptualization, but rather something we are hard-wired with at birth, EVEN when it is also possible to learn to conceptualize that idea through general learning, and EVEN when we don't necessarily even use recursion? If this is what he thinks, he has not given any reason as to what exactly is his motivation to believe this way. What possible reason would there be for us to have an innate ability to understand language recursion, when it is clear we don't even have an innate ability to understand colors (as in, blind people cannot fathom the concept of colors).

 

Or perhaps he is saying that it is impossible to learn the idea of recursion via general world modeling, but rather it MUST be a hard-wired ability. Well, is not the theory of universal grammar itself an instance of general learning that has conceptualized recursion? Or is that theory also somehow hard-wired in us, "whether we employ it or not?" Hello?!

 

Anyway, just so this would not end up being just a complete bashing towards Noam Chomsky, let's bash Isaac Asimov little bit too. :) It is the whole point of general learning system that it is capable of building a world view on its own; we don't program in it what to do, we program in how to build predictive models. Practically this means the system necessarily builds a belief system, and it is not very hard to imagine scenarios where individual AI systems build a belief system where human beings are not exactly the top dog. There's no reason to believe this would happen to all AI systems in the world all of a sudden, just like there's no reason to believe all the human beings on earth would suddenly come to build identical belief systems. But in principle nothing prevents us from still hard wiring reward systems where the AI simply very much likes to perceive happiness in people, or they simply very much like to do whatever it is what we want them to do (like in Hitchhiker's Guid to Galaxy the doors that just very much like to open and close for human beings).

 

But even if we do, it is trivial to imagine situations where a general learning system simply makes a mistake (its belief system is always wrong to an extent), or where in order to avoid harm to someone means harming someone else.

 

I think this talk about programming robots to not harm human beings is just another case of naive perspective on what "intelligence" itself is.

 

-Anssi