Discussion of philosophy
Will be forever beholden to his work on language and grammar. Made it bearable to sit through a pointless CompSci class.
I wont forgive him for torturing me with his pointless UG shit.
I thought his language theories were disproven?
The Chomsky hierarchy is still taught in computer science. That's as much as I know of Chomsky. I don't know anything about his philosophy or natural language contributions
Such a towering figure in the field of linguistics, politics and philosophy of psychology. Especially his linguistics work was something I found personally extremely engaging as a student. Reading his review of B.F. Skinner's book was a highlight of my philosophy of psychology class.
I know he has been active since the turn of the century but his influence was definitely more of a 20th century phenomenon. Not many academics reach the level of popular awareness that he did.
For you yongsters out there, Chomsky vs Skinner was the original Kendrick Lamar vs Drake.
Chomsky vs Skinner was the original Kendrick Lamar vs Drake.
This comparison is extra ironic because their argument was over language acquisition and neither Lamar nor Drake can read.
Who is Drake?
Contemporary Phenomenologist with radical views about sexual liberation. Basically a latter day Foucault.
The guy from the meme.
Very sad to see our mortality as we watch Chomsky slip away. Let's celebrate his life and his accomplishments - our eventual deterioration and demise is all a part of our grand story of life. He's done amazing things with his time here and we'd all be lucky to have 95 great years like that.
Let's celebrate his life and his accomplishments
I'm a big fan of his work in the field of genocide denial and atrocity apologetics, truly inspirational.
Such an interesting and weird life to have lived. Academically, he lived to see almost his entire body of work thoroughly disrupted by LLMs, specifically the concept of an universal grammar/ and the idea of an 'innate language acquisition device'. I was in a live stream with him about 2 months or so after the first beta models from open AI had started to poke their way into the main-stream. It felt kind of sad, because its like, he was obviously very long in the tooth even then, but when your entire academic career is based on like "one thing", as so many scientists and philosophers careers are, when that 'one thing' ends up being demonstrably false, it seems kind of.. soul crushing? I saw it once before when the revised genomic classification of plants was being released and one of my professors (who i think was like, 1-2 years out from retirement), watched his life time body of work of taxonomic classification get "yeah nah dawged" by the revised genomic taxonomy. I'll say he was not the most engaged instructor and actually insisted on teaching us wrong, which was super aggravating.
I still think Chomskys political work stands apart as a life well lived.
This comment is global-warming-denialism levels of stupid. I’m honestly shocked.
LLM’s have no such implications for the field of linguistics. They’re barely relevant at all.
Do I really need to point out that human beings do not learn language the way LLMs “learn” language? That human beings do not use language the way LLM’s use language? Or that human beings are not mathematical models. Not even approximately. I fucking hate this timeline.
Thank you for saying it. That really was a depressingly incurious comment.
Chomsky, Ian Roberts and Jeffrey Watumull on the topic: https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html
Suppose you are holding an apple in your hand. Now you let the apple go. You observe the result and say, “The apple falls.” That is a description. A prediction might have been the statement “The apple will fall if I open my hand.” Both are valuable, and both can be correct. But an explanation is something more: It includes not only descriptions and predictions but also counterfactual conjectures like “Any such object would fall,” plus the additional clause “because of the force of gravity” or “because of the curvature of space-time” or whatever. That is a causal explanation: “The apple would not have fallen but for the force of gravity.” That is thinking.
I'm not saying this from a defending LLMs point, I genuinely think they are a waste of so many things in this world and I can't wait for this hype cycle to be over.
However, there is a lot of research and backing behind statistical learning in language acquisition, this is specifically the research subject of my friends. It's a very big thing in intervention for delays in language.
It is opposed to Chomsky's innate language theory, which at this point I think almost any linguist or language/speech sciences researcher would tell you isn't a well accepted theory (at least as a holistic explanation, certainly it could still be true to an extent and a part of other systems).
tl;Dr LLMs are stupid, but it's not broadly true that the way they "learn language" is entirely different from how humans do. The real difference is that they fail to actually learn anything even when imitating humans.
Be that as it may, we aren’t getting any answers from LLM’s. And given that Universal Grammar was the dominant view for so long, the jury is still out on a viable alternative.
This would be true if chomskys claim was that he was simply studying human language acquisition and that machines are different, but his claim was that machines can't learn human languages because they don't have some intuitive innate grammar.
Saying an llm hasn't learned language becomes harder and harder the more you talk to it and the more it starts walking like a duck and quacking like a duck. To make that claim you'll need some evidence to counter the demonstrable understanding the llm displays. Chomsky in his nytimes response just gives his own unprovable theories on innate grammar and some examples of questions llms "can't answer" but if you actually ask any modern llm they answer them fine.
You can define "learning" and "understanding" in a way that excludes llms but you'll end up relying upon unprovable abstract theories until you can come up with an example of a question/prompt that any human would answer correctly and llms won't to demonstrate that difference. I have yet to see any such examples. There's plenty of evidence of them hallucinating when they reach the edge of their understanding, but that is something humans do as well.
Chomsky is still a very important figure and his work on politics with manufacturing consent is just as relevant as when it was written over 20 years ago. His work on language though is on shaky grounds and llms have made it even shakier.
Do you really think Chomsky’s UG hypothesis from half a century ago was formulated to deny that some dumb mathematical model would be able to simulate human speech?
Almost nothing you’ve written has any grounding in empirical reality. You have a sentence that reads something like “the more you talk to LLM’s the harder it is to deny that they can use and understand language.”
You might as well say that the longer you stare at a printed painting, the harder it is to deny that printers make art. LLM’s do not “understand” their outputs or their inputs. If we feed them nonsense, they output nonsense. There’s no underlying semantics whatsoever. An LLM is a mathematical model.
I know it looks like magic, but it’s not actually magic. And even if it were, it would have nothing to do with linguistics, which is concerned with how humans, not computers, understand and manipulate language. This whole ridiculous conversation is a non-sequitur.
You are like, science denial-ism level of ignorance when it comes to this conversation, or perhaps, it might be that you don't actually understand the underlying philosophy of scientific inquiry to understand why LLMs basically were able to break the back of both UH and innate acquisition.
You seem like the kind of person who cheer-leads "in the spirit of science" but doesn't actually engage in it as a philosophical enterprise. You didn't seem to notice the key point that @Not_mikey@slrpnk.net made, which is that unlike UG, LLM's are actually testable. That's the whole thing right there, and if you don't get the difference, that's fine, but it speaks to your level of understanding to how one actually goes about conducting scientific inquiry.
And if you want to talk about incurious:
You might as well say that the longer you stare at a printed painting, the harder it is to deny that printers make art. LLM’s do not “understand” their outputs or their inputs. If we feed them nonsense, they output nonsense. There’s no underlying semantics whatsoever. LLM’s are a mathematical model.
Specifically "There’s no underlying semantics whatsoever" is the key lynch pin that UG demands that LLM's demonstrate are not strictly necessary. Its exactly why Chomskys house of cards crumbles with the the counter-factual to UG/ Innate acquisition that LLM's offer. I had a chance to ask him this question directly about 6 months prior to that op-ed being published. And he gave a response that's about as incurious about why LLM, and basically, big complex networks in general, are able to learn as you've offered here. His was response was basically the same regurgitation on UG and innate acquisition that he offers in the op-ed. And the key point is that yes, LLM's are just a big bucket of linear algebra; but they represent an actually testable instrument for learning how a language might be learned. This is the most striking part of Chomskys response and I found it particularly galling.
And it is interesting that yes, if you feed ~~LLM's~~ transformers (I'm going to start using the right term here: transformers) unstructured garbage, you get unstructured garbage out. However, if there is something there to learn, they seem to be at least some what effective at finding it. But that occurs in non-language based systems as well, including image transformers, transformers being used to predict series data like temperature or stock prices, even even DNA and RNA sequences. We're probably going to be having transformers capable of translating animal vocalizations like whale and dolphin songs. If you have structured series data, it seems like transformers are effective at learning patterns and generating coherent responses.
Here's the thing. Chomsky UG represented a monolith in the world of language, language acquisition and learning, and frankly, was an actual barrier to progress in the entire domain, because we now have a counter factual where learning occurs and neither UG or innate acquisition are necessary or at all relevant. Its a complete collapse of the ideas, but its about as close as we'll get because at least in one case of language acquisition, they are completely irrelevant.
And honestly, if you can't handle criticism of ideas in the sciences, you don't belong in the domain. Breaking other peoples ideas is fundamental the process, and its problematic when people assume you need some alternative in place to break someone elses work.
the key lynch pin that UG demands that LLM's demonstrate are not strictly necessary
You know what, I’m going to be patient. Let’s syllogize your argument so everyone can get on the same page, shall we.
This argument is not valid, because it’s missing at least one premise. Once you come up with a valid argument, we can debate its premises. Until then, I can’t actually respond, because you haven’t said anything substantive.
The mainstream opinion in linguistics is that LLM’s are mostly irrelevant. If you believe otherwise — for instance, that LLM’s can offer insight into some abstract UG hypothesis about developmental neurobiology — explain why, and maybe publish your theory for peer review.
Universal grammar isn’t “almost his entire body of work”—for the last thirty years he’s been at the forefront of the Minimalist program which approaches syntax from a very different direction. (And for twenty years prior to that he was working on X-bar theory, which was also a departure from universal grammar.) He’d still be considered one of the leading linguists of the past century on the strength of his non-UG work alone.
And Chomsky’s criticism of LLMs as a model of the brain’s internal language process is absolutely valid: while LLMs can imitate human languages, they can just as readily imitate unnatural constructed languages humans would never organically create. So they’re (at best) post-hoc predictive models, not explanatory ones—like Ptolemy’s epicycles, which could accurately predict the motions of the known planets but couldn’t generally distinguish between physically possible orbits and ones that would violate Newtonian mechanics, and thus provided no real insight into the natural world.
I'd say his language theories had already been sufficiently challenged way before the advent of LLMs. But it's hard to deny that they helped to lay the groundwork for what we know about linguistics today, even if they weren't proven out in the end.
Two of my favorite Chomsky moments are his interview with Brian McGee and his debate with Foucault.
He has also written extensively about political ideologies, economics, and other issues. Linguistics is far from the entirety of his academic accomplishments.
It doesn’t sound like there’s necessarily been any recent change in his condition—their ultimate source is a month-old Reddit post from his former assistant, who was relaying information she’d heard from his family a month prior to that.
Edit: This AP story has more info: he hasn’t been able to speak since a stroke last year, but he was able to travel to Brazil with his wife via hospital jet and is still following the news from Gaza.
A few months is still recent for people not following his health closely.
True, but even the info from the family member was just confirming that his condition hadn’t improved since the “medical event” he suffered a full year ago.
You changed my life Mister Chomsky. Thank you
We're all lucky he said as much as he did while he could speak.
Oh hey, I saw something by accident that I can contribute here: https://www.youtube.com/watch?v=KWKQIqzotLQ
It's a video response from Chomsky's current collaborators telling off these journalists for announcing a private health matter to the public & making it harder for Chomsky & his family & emphasising that even now into his 90s he is doing cutting-edge much-discussed intellectual work and that is the real news.
so who do we email now?
Nonononono