Seventy years ago, Noam Chomsky helped revolutionize linguistics by arguing that human language is fundamentally computational. Today, artificial intelligence is demonstrating just how powerful computational language systems can be—while raising uncomfortable questions about some of the theories that helped inspire them.
The Linguist and the Machine
There's a rather delicious irony lurking in the history of artificial intelligence.
In the 1950s, a young linguist named Noam Chomsky helped transform our understanding of human language by arguing that it wasn't simply a collection of learned expressions or habitual responses. Instead, he proposed that language was generated by an internal computational system capable of producing an effectively unlimited number of sentences according to abstract grammatical principles.
It was a revolutionary idea.
Language, in this view, wasn't merely something we learned to imitate. It was something our minds could compute.
Fast-forward nearly seventy years.
We've now built machines capable of producing remarkably sophisticated language. They write essays, translate between languages, explain complicated ideas, generate computer programs, and occasionally compose poetry that is either surprisingly moving or spectacularly terrible.
And they accomplish much of this without anyone explicitly programming the grammatical rules that Chomsky and generations of linguists spent decades investigating.
Instead, these machines learn statistical relationships from enormous collections of text.
Which creates a fascinating historical twist.
Chomsky helped establish the importance of computational explanations of language. Yet some of the most successful computational language systems ever created operate according to principles quite different from those he proposed for the human mind.
Did Chomsky accidentally predict ChatGPT?
Not exactly.
But the relationship between generative linguistics and generative artificial intelligence is considerably more interesting than the coincidence of their names.
And understanding that relationship may tell us something important about both human language and artificial intelligence.
Language Is Not a Collection of Sentences
To understand the connection, we need to return to 1957.
That year, Chomsky published Syntactic Structures, a relatively short book that helped launch a revolution in theoretical linguistics.
At the time, American structural linguistics largely emphasized the systematic description of observable language patterns. Meanwhile, behaviorist approaches to psychology emphasized learning through environmental experience, conditioning, and reinforcement.
Chomsky challenged the adequacy of such approaches for explaining the creative nature of human language.
Consider a simple sentence:
The dog chased the cat.
Now consider a somewhat more elaborate one:
The dog that chased the cat that frightened the mouse that ate the cheese was barking.
You've probably never encountered that exact sentence before.
Nevertheless, you understand it.
You know which animal was barking. You know which animal frightened the mouse. You can identify the relationships among the various creatures involved in this increasingly unfortunate sequence of events.
And you can construct countless similar sentences without having heard them previously.
This creative capacity was central to Chomsky's argument.
Human language isn't simply a repository of memorized expressions. It is a system that allows us to generate novel sentences from a finite collection of grammatical principles.
In other words, language is productive because it is computational.
That word—computational—is important.
Chomsky wasn't suggesting that the human brain contains a little digital computer running grammatical software. Rather, he was proposing that linguistic knowledge could be understood in terms of formal operations and structural relationships.
Those operations could be described mathematically.
And once language became a subject of formal computational investigation, it helped create an intellectual foundation for fields including computational linguistics, natural language processing, and cognitive science.
Chomsky wasn't the sole architect of these developments, of course. Modern artificial intelligence emerged from many intellectual traditions, including mathematics, statistics, information theory, computer science, and neuroscience.
But his work helped establish a crucial idea: language could be investigated as a formal computational system rather than merely cataloged as a collection of observed utterances.
That idea would prove enormously influential.
Colorless Green Ideas and the Problem with Statistics
Chomsky also introduced one of the most famous examples in the history of linguistics:
Colorless green ideas sleep furiously.
It's a wonderfully absurd sentence.
Ideas don't ordinarily possess colors. Something cannot generally be both colorless and green. And the notion of an idea sleeping furiously suggests a particularly troubled philosophical existence.
Nevertheless, the sentence is grammatically well formed.
Compare it with:
Furiously sleep ideas green colorless.
The second sentence contains precisely the same words, but their arrangement violates the ordinary grammatical structure of English.
Chomsky used this contrast to demonstrate that grammaticality cannot simply be equated with meaningfulness, familiarity, or the frequency of particular word sequences.
A sentence can be grammatically correct while being semantically bizarre.
And a sequence of perfectly ordinary words can be grammatically unacceptable.
This distinction helped motivate Chomsky's criticism of approaches that attempted to explain linguistic knowledge through statistical properties of observed language alone.
To be clear, the argument wasn't that statistical information is irrelevant to language. Rather, Chomsky argued that statistical associations, particularly the relatively simple ones available to researchers at the time, were insufficient to explain the abstract structural knowledge possessed by human speakers.
For decades, this helped define a major intellectual divide.
On one side were researchers investigating symbolic grammatical rules and hierarchical structures.
On the other were researchers exploring statistical approaches to language processing and acquisition.
And then artificial intelligence began doing something neither side could comfortably ignore.
What If We Just Let the Machine Read Everything?
Modern large language models approach language from a very different direction than traditional generative grammar.
Instead of beginning with an explicitly defined collection of grammatical rules, they learn from enormous quantities of linguistic data.
At the heart of many contemporary language models is a deceptively simple training objective:
Given the text that has come before, predict what comes next.
Consider:
The cat sat on the...
There's a reasonably good chance that the next word will be mat.
That seems straightforward enough.
But predicting language accurately becomes much more complicated when sentences involve nested phrases, ambiguous meanings, long-distance dependencies, and relationships among words separated by substantial amounts of intervening material.
Consider these two sentences:
The keys to the cabinet are missing.
The key to the cabinets is missing.
In the first sentence, the verb is plural because the subject is keys.
In the second, the verb is singular because the subject is key.
Notice that the noun immediately preceding the verb is misleading in both cases.
A simplistic statistical model relying on nearby word associations might choose the wrong verb.
But a sufficiently capable language model can learn to account for the hierarchical relationship between the subject and its verb.
And that's where things become interesting.
Modern neural language models can acquire representations that reflect grammatical relationships without being explicitly programmed with traditional syntactic rules.
Researchers have even demonstrated that aspects of syntactic structure can be recovered from the internal numerical representations of these models.
In other words, when we train a machine to predict linguistic sequences, it can develop internal representations that encode information about grammatical hierarchy.
We didn't necessarily teach it what a noun phrase was.
But it learned patterns that allow it to behave as though it recognizes many of the relationships noun phrases participate in.
And that raises an intriguing possibility.
Perhaps some of the grammatical structures described by linguists are also structures that powerful learning systems naturally discover when trying to make sense of language.
The Great Irony: Chomsky Was Right About Structure, But What About Learning?
Here's where the historical irony becomes especially compelling.
Chomsky's work helped establish that language involves abstract computational structure.
Modern artificial intelligence has provided striking demonstrations of how sophisticated computational language processing can become.
But these demonstrations also complicate some of Chomsky's arguments about how grammatical knowledge is acquired.
One of the central ideas associated with generative linguistics is the poverty of the stimulus.
The argument begins with a remarkable observation.
Human children acquire extraordinarily sophisticated grammatical knowledge despite receiving limited, imperfect, and largely unstructured linguistic input.
Nobody sits a toddler down and explains long-distance syntactic dependencies.
Parents don't ordinarily provide formal lessons on the constraints governing grammatical movement.
Yet children acquire linguistic intuitions that extend far beyond the specific sentences they've encountered.
Chomsky argued that this suggests human beings possess an innate linguistic endowment that constrains the kinds of grammatical systems they can acquire.
This idea became associated with Universal Grammar.
The precise content of Universal Grammar has changed substantially across different versions of generative theory. Some formulations propose relatively rich innate linguistic structure, while others seek to minimize the language-specific machinery required.
But the underlying question remains:
How do children acquire such sophisticated grammatical knowledge from the evidence available to them?
Now consider what we've accomplished with artificial intelligence.
We've built systems that can learn extraordinarily complex linguistic patterns without explicitly programming them with a traditional grammar.
Does that mean the poverty-of-the-stimulus argument has collapsed?
Not so fast.
Modern language models typically consume vastly more linguistic material than any child could encounter during language acquisition.
They also possess architectural features that influence what kinds of patterns they can learn.
And children acquire language while interacting with people, objects, intentions, emotions, and the physical world—not simply by processing written text.
So the success of a language model doesn't demonstrate that human children acquire grammar through identical mechanisms.
But it does challenge the assumption that sophisticated grammatical behavior necessarily requires explicitly language-specific grammatical rules to be built into a learning system.
And that is a significant development.
The interesting question is no longer simply whether grammar is learned or innate.
It's how much grammatical structure must be supplied by the learner's cognitive architecture, and how much can emerge from experience.
Did Chomsky Help Create the Very Thing That Challenges Him?
There's an important distinction to make here.
Chomsky did not invent neural networks, transformer architectures, or modern large language models.
And the technical lineage of ChatGPT runs primarily through developments in machine learning, statistics, neural computation, and computer science rather than through direct implementations of Chomskyan grammar.
Indeed, Chomsky has been publicly critical of large language models and of attempts to equate their statistical capabilities with human linguistic understanding.
Nevertheless, the broader intellectual history contains a genuine connection.
Chomsky helped make the computational nature of language a central scientific question.
He demonstrated that linguistic theory could be expressed through formal systems and investigated using mathematical principles.
He helped establish a framework in which researchers could ask what kinds of computational mechanisms are capable of generating and interpreting human language.
Later researchers pursued that question using a wide variety of approaches, including methods quite different from Chomsky's own.
And eventually, some of those approaches produced systems capable of generating language with a sophistication that would have seemed extraordinary in 1957.
So did Chomsky predict ChatGPT?
No, not in any meaningful technical sense.
But he helped establish some of the fundamental questions that systems like ChatGPT now force us to reconsider.
There's something wonderfully appropriate about that.
Scientific revolutions frequently produce discoveries that complicate the theories that helped make them possible.
Newtonian mechanics helped establish the mathematical foundations of modern physics, even though later developments revealed limitations in Newton's account of the universe.
Likewise, the computational study of language has evolved in directions that challenge some of its own foundational assumptions.
That's not a failure of science.
That's science doing exactly what it's supposed to do.
Syntax Trees Versus Statistical Clouds
Traditional generative syntax describes language through hierarchical structures.
Sentences contain phrases. Phrases contain smaller constituents. Those constituents combine according to grammatical principles.
In Chomsky's Minimalist Program, a central operation called Merge combines syntactic objects into larger structures.
The underlying idea is elegant: perhaps much of the extraordinary complexity of human language emerges from the repeated application of a relatively simple computational operation.
Modern transformers work differently.
They process numerical representations of tokens through layers of mathematical operations.
One of their defining mechanisms, attention, allows information associated with one token to be influenced by information associated with others.
Through repeated processing, the model develops increasingly sophisticated contextual representations.
But it doesn't necessarily construct explicit syntactic trees.
There may be no identifiable symbolic object corresponding to a noun phrase, no explicit grammatical movement operation, and no neatly organized collection of rules resembling those in a linguistics textbook.
Nevertheless, the model can often produce language that conforms to the structural generalizations those rules describe.
Which raises a fascinating question.
Does the model possess grammar?
Or has it simply learned a sufficiently complicated statistical approximation of grammatical behavior?
And perhaps more importantly, how would we distinguish those possibilities?
If a neural network consistently recognizes hierarchical dependencies, applies grammatical generalizations to unfamiliar sentences, and distinguishes acceptable from unacceptable constructions, then dismissing its behavior as mere statistics begins to feel inadequate.
After all, statistical learning can produce abstract representations.
And abstract representations can support behavior that looks remarkably rule-governed.
But the opposite caution matters too.
Just because a model behaves as though it possesses a grammatical rule doesn't mean its internal computations resemble those of a human speaker.
Similar outputs do not establish identical mechanisms.
That distinction is crucial if we're interested in using artificial intelligence to understand human cognition.
What Does It Actually Mean to Know Grammar?
Suppose a language model correctly identifies the grammatical subject of a sentence.
Suppose it recognizes syntactic ambiguity.
Suppose it can explain why one sentence is acceptable while another is not.
Does it understand grammar?
That depends partly on what we mean by understand.
From a behavioral perspective, the model demonstrates substantial grammatical competence.
But from a cognitive perspective, we still need to investigate what representations and computational mechanisms produce that behavior.
Consider an analogy.
A calculator and a human mathematician can arrive at the same answer to an arithmetic problem.
That doesn't mean they solve the problem using identical cognitive processes.
Similarly, a neural language model and a human speaker may produce the same grammatical judgment while relying on very different internal mechanisms.
And the differences become particularly revealing when the systems fail.
Language models sometimes struggle with unusual syntactic constructions, complicated long-distance dependencies, or carefully designed examples that distinguish hierarchical generalization from familiar surface patterns.
Humans make grammatical errors too, but the patterns of those errors aren't necessarily the same.
These differences provide clues about what the systems have actually learned.
In fact, this brings us back to another important Chomskyan distinction: competence versus performance.
Human linguistic competence refers to the underlying grammatical knowledge possessed by a speaker.
Performance refers to the actual use of that knowledge, subject to memory limitations, distractions, processing difficulties, and ordinary human fallibility.
When an AI produces an ungrammatical sentence, what exactly has gone wrong?
Does it lack the relevant grammatical generalization?
Does it possess a useful representation but fail to apply it consistently?
Or does the probabilistic generation process simply produce an unlikely or undesirable continuation?
These questions are not easy to answer.
And they illustrate why studying the behavior of language models is only the beginning of understanding them.
The Experiment That Could Change the Conversation
For me, the most compelling implication of modern AI isn't that machines can write convincing prose.
It's that they provide an entirely new way to investigate longstanding theories about language acquisition.
Imagine building a language model that receives something approximating the linguistic experience of a human child.
Not trillions of words harvested from the internet.
Not the accumulated written output of civilization.
Instead, give it a developmentally realistic amount of language, presented in a realistic sequence, alongside some representation of the social and physical context in which language occurs.
Then examine what it learns.
Does it acquire hierarchical syntax?
Can it generalize grammatical rules to unfamiliar constructions?
Does it recognize constraints that were never explicitly explained?
Does it develop grammatical knowledge at roughly the same stages as a human child?
And perhaps most revealingly, does it make the same kinds of mistakes?
Researchers are already exploring related questions through initiatives such as the BabyLM Challenge, which investigates language learning under more constrained and developmentally motivated training conditions.
These experiments won't automatically settle the nature-versus-nurture debate.
Even a successful model might rely on architectural biases that differ substantially from those of the human brain.
And a model's failure to acquire a particular grammatical generalization wouldn't necessarily establish that humans possess an innate rule for it.
Nevertheless, such experiments offer something genuinely exciting.
They allow researchers to investigate the relationship between linguistic input, learning mechanisms, and grammatical knowledge through controlled computational experiments.
Questions that once seemed almost entirely theoretical can now be explored through systems that actually learn and generate language.
That's a remarkable development.
Maybe the Rules Were in the Patterns All Along
There's a temptation to frame the relationship between generative linguistics and artificial intelligence as a contest.
Chomsky versus ChatGPT.
Rules versus statistics.
Nature versus nurture.
But these oppositions may be misleading.
A statistical learning system can acquire representations that encode abstract structural regularities.
Those regularities can function much like grammatical rules, even if nobody explicitly programmed them.
Conversely, the fact that a symbolic grammar elegantly describes a linguistic phenomenon doesn't establish that the human brain stores or processes that grammar in precisely the same form.
Perhaps the two approaches are investigating different aspects of the same underlying phenomenon.
Generative syntax attempts to characterize the structural principles governing linguistic expressions.
Neural language modeling investigates how sophisticated linguistic behavior can emerge through learning and computation.
Neither perspective necessarily eliminates the need for the other.
In fact, the success of modern language models may reinforce one of Chomsky's most important insights: language involves abstract structural relationships that extend far beyond simple associations between neighboring words.
What remains unsettled is where those relationships come from.
Are they the expression of a specialized biological system shaped by evolution?
Are they emergent properties of powerful general-purpose learning mechanisms?
Or do they arise through some interaction among inherited cognitive constraints, statistical learning, and the structure of human experience?
We don't yet know.
And perhaps that's the most interesting part of the story.
The Machine Comes Full Circle
In 1957, Chomsky helped establish a vision of language as a computational system capable of generating an unlimited range of expressions through abstract structural principles.
In the decades that followed, researchers developed increasingly sophisticated computational approaches to language.
Some pursued explicit grammatical rules.
Others pursued statistical learning.
Still others combined the two.
Eventually, advances in neural networks and machine learning produced systems capable of generating language with extraordinary fluency.
And now those systems are forcing us to reconsider questions that have occupied linguists for generations.
How much grammatical knowledge must be innate?
How much can be learned?
What does it mean to possess a rule?
And how closely must a computational model resemble the human mind before it can tell us something meaningful about human cognition?
Chomsky certainly didn't predict ChatGPT in the ordinary sense.
And modern language models haven't rendered generative linguistics obsolete.
But there's an undeniable historical symmetry.
A linguistic revolution helped establish the computational study of language.
That broader scientific enterprise eventually produced machines capable of generating sophisticated language.
And those machines are now becoming experimental tools for investigating the very nature of grammatical knowledge.
It's a wonderfully circular development.
We began by asking what kind of computational system could explain human language.
We built computational systems that could produce language.
And now we're asking whether those systems can help explain the humans who inspired them.
We set out to teach machines to speak like us. We may end up learning something fundamental about why we can speak at all.
References and Further Reading
The following works provide historical, theoretical, and empirical background for the discussion of generative grammar, statistical language learning, and artificial intelligence.
-
Chomsky, N. (1957). Syntactic Structures. Mouton.
The foundational work that helped establish transformational generative grammar and introduced the famous "colorless green ideas" example.
-
Chomsky, N. (1965). Aspects of the Theory of Syntax. MIT Press.
Publisher information.A landmark development of generative grammar, including the distinction between linguistic competence and performance.
-
Chomsky, N. (1995). The Minimalist Program. MIT Press.
Publisher information.Introduces the minimalist framework, which seeks to explain linguistic structure through a more economical computational system.
-
Linzen, T., Dupoux, E., & Goldberg, Y. (2016). Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4, 521–535.
Read the paper.Investigates whether neural networks can learn grammatical relationships such as subject-verb agreement without explicit syntactic representations.
-
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30.
Read the paper.The influential paper introducing the transformer architecture that became foundational to modern large language models.
-
Gulordava, K., Bojanowski, P., Grave, E., Linzen, T., & Baroni, M. (2018). Colorless green recurrent networks dream hierarchically. Proceedings of NAACL-HLT 2018, 1195–1205.
Read the paper.Demonstrates that recurrent neural networks can acquire important aspects of hierarchical grammatical structure, including in semantically nonsensical sentences.
-
Hewitt, J., & Manning, C. D. (2019). A structural probe for finding syntax in word representations. Proceedings of NAACL-HLT 2019, 4129–4138.
Read the paper.Explores how information about syntactic trees can be recovered from the internal representations of neural language models.
-
Warstadt, A., Mueller, A., Choshen, L., Wilcox, E., Zhuang, C., Ciro, J., Mosquera, R., Paranjabe, B., Williams, A., Linzen, T., & Cotterell, R. (2023). Findings of the BabyLM Challenge: Sample-efficient pretraining on developmentally plausible corpora. Proceedings of the BabyLM Challenge at CoNLL 2023, 1–34.
Read the paper.Investigates how effectively language models can learn from smaller, developmentally motivated training datasets.
-
Piantadosi, S. T. (2024). Modern language models refute Chomsky's approach to language. In E. Gibson & M. Poliak (Eds.), From Fieldwork to Linguistic Theory: A Tribute to Dan Everett (pp. 353–414). Language Science Press.
Read the chapter.A provocative argument that the success of modern language models challenges fundamental assumptions of Chomskyan linguistics, particularly strong claims about innate grammatical knowledge.
These references represent several perspectives on the relationship between linguistic structure, learning, and computation. The field remains an active area of scientific debate, and the success of neural language models does not, by itself, establish that human language acquisition operates through the same mechanisms.