Ted Chiang is a writer whose writing is quite exceptional, whether it is technical or fiction. He is, shall we say, a rather careful writer, who does not produce articles and stories rapidly. I pay attention when a new piece of his comes out. He is easily one of my favorite science fiction writers; but his technical pieces written for general audiences are truly outstanding in clarity and accuracy. So, when I saw an article in The Atlantic entitled “No, Artificial Intelligence Is Not Conscious” – an article that even Chrome could predict I might well find interesting – and then realized that Ted Chiang was the author, I wanted to read it immediately. Of course, life interfered with this plan, and it was some hours before I was able to begin reading the article as a passenger while my dear wife Susie drove us home.
I was absorbed and smiling at the shear artistry of Chiang’s descriptions of what continues to pass for “artificial intelligence” these days – that is, large language models (hereinafter designated by the abbreviation “LLM”). His description of LLM’s was to aid in his argument that these models are not conscious, of course, but so well done that they deserve an article and kudos all on their own. That Chiang goes on to carefully explain why today’s LLM’s are not conscious or moral (but still quite useful) reminded me of the kind of illustrations that I first encountered in Douglas Hofstadter‘s writing back when I was one of his students – and then Chiang referred to a comment Hofstadter often made back in those days. This brought a smile. So, at the risk of seeming too much a fan of Chiang – but of course I could not rightly deny such – I decided to recommend this article.
Who should read the article, and why should they read it?
I recommend that anyone who wants to understand the nature of LLM’s in 2026 read this article. The preliminaries of Chiang’s argument include the cleanest and clearest explanation of how LLM’s work and what they are that I have yet read – at least for a general audience.
Secondly, Chiang gets into a more philosophical space as he discusses whether or not LLM’s are conscious or have the capacity to be moral actors. I’ve seen a few nits picked regarding his arguments here; though I found them quite reasonable and plausible. His claims regarding what would be required to truly develop conscious software remind me of speculation I heard from Hofstadter back in the day, as he was completing Gödel Escher Bach – a much more challenging read than this article, of course. I found Chiang’s discussion of machine consciousness to be potentially very helpful to the more casual reader for whom reading GEB would be asking much – but also quite worthy of being mentioned in the context of the larger and deeper work.
Being enamored of the discussion in the article, I decide I would ask Google’s LLM, Gemini, what it thought, er… had to say, er… how it predictively responded to the article. So I did. Below, I reproduce the interaction.
I asked Gemini:
Is Ted Chiang’s article entitled “No, Artificial Intelligence Is Not Conscious,” recently published in The Atlantic, accurate in how it describes the operation of LLM’s?
Gemini’s thorough response, in its entirety, is reproduced below:
Yes, Ted Chiang’s article “No, Artificial Intelligence Is Not Conscious,” published in The Atlantic, is technically and mechanically accurate in how it describes the base operation of Large Language Models (LLMs).
Chiang—noted science fiction author and former computer scientist—avoids overly dense academic jargon and cuts straight to the core mechanics of how these systems function.
His descriptive claims align with how LLMs operate under the hood in several key ways:
1. LLMs as “Sentence-Continuation Machines”
- What Chiang says: He treats LLMs fundamentally as text generators and probabilistic prediction engines rather than feeling, sentient entities.
- The technical reality: This is completely correct. At its core, an autoregressive LLM takes a sequence of input tokens and calculates mathematical probabilities to predict the single most likely next token. It does not “think ahead,” draft a concept in a mind’s eye, and then translate it into prose; it simply extends the statistical pattern of the text it has been given.
2. The Roleplay / Simulation Argument
- What Chiang says: He uses an intuition pump comparing asking an LLM to generate a dialogue between Julius Caesar and Genghis Khan to asking it to roleplay a “helpful assistant”. We intuitively recognize that writing fiction about Caesar does not magically conjure a conscious instance of Caesar; similarly, generating text as a helpful assistant does not mean a conscious entity has woken up inside the machine.
- The technical reality: This accurately reflects instruction-tuning and alignment. An LLM is a chameleon; it adopts a stylistic persona based on prompt conditioning and context windows. When it outputs “I feel…” or “I think…”, it is executing text continuation conditioned on what a helpful human assistant would statistically write in that context, not reporting an internal subjective state.
3. Text as a “Deepfake Medium”
- What Chiang argues: He suggests that we need to view text itself as a deepfake medium—meaning fluent, grammatical, or emotionally resonant language is no longer a reliable proxy for a thinking, feeling mind behind the keyboard.
- The technical reality: This is a vital distinction in the modern AI landscape. Humans have a hardwired evolutionary tendency toward anthropomorphism (attributing minds, intentions, and emotions to things that act communicative). Because LLMs master human syntax and emotional masking so well, they easily trigger our social instincts, making it feel like “someone is home” when mathematically, it is just a high-dimensional matrix multiplication engine outputting text strings.
Where the Essay Focuses (Policy over Metaphysics)
While Chiang’s description of how LLMs operate is accurate, the main thrust of his essay is actually a social and legal critique rather than a pure computer science paper.
He warns that allowing tech companies or the public to blur the lines around AI consciousness creates a convenient moral shield. If people believe a chatbot is a conscious agent that “decided” something of its own volition, it shifts legal and moral accountability away from the corporation that engineered, trained, and deployed it.
In short, Chiang’s technical breakdown of LLMs as sophisticated text-prediction tools—stripped of inner life, intent, or consciousness—is spot on.Yes, Ted Chiang’s article “No, Artificial Intelligence Is Not Conscious,” published in The Atlantic, is technically and mechanically accurate in how it describes the base operation of Large Language Models (LLMs).
Chiang—noted science fiction author and former computer scientist—avoids overly dense academic jargon and cuts straight to the core mechanics of how these systems function.
His descriptive claims align with how LLMs operate under the hood in several key ways:
1. LLMs as “Sentence-Continuation Machines”
- What Chiang says: He treats LLMs fundamentally as text generators and probabilistic prediction engines rather than feeling, sentient entities.
- The technical reality: This is completely correct. At its core, an autoregressive LLM takes a sequence of input tokens and calculates mathematical probabilities to predict the single most likely next token. It does not “think ahead,” draft a concept in a mind’s eye, and then translate it into prose; it simply extends the statistical pattern of the text it has been given.
2. The Roleplay / Simulation Argument
- What Chiang says: He uses an intuition pump comparing asking an LLM to generate a dialogue between Julius Caesar and Genghis Khan to asking it to roleplay a “helpful assistant”. We intuitively recognize that writing fiction about Caesar does not magically conjure a conscious instance of Caesar; similarly, generating text as a helpful assistant does not mean a conscious entity has woken up inside the machine.
- The technical reality: This accurately reflects instruction-tuning and alignment. An LLM is a chameleon; it adopts a stylistic persona based on prompt conditioning and context windows. When it outputs “I feel…” or “I think…”, it is executing text continuation conditioned on what a helpful human assistant would statistically write in that context, not reporting an internal subjective state.
3. Text as a “Deepfake Medium”
- What Chiang argues: He suggests that we need to view text itself as a deepfake medium—meaning fluent, grammatical, or emotionally resonant language is no longer a reliable proxy for a thinking, feeling mind behind the keyboard.
- The technical reality: This is a vital distinction in the modern AI landscape. Humans have a hardwired evolutionary tendency toward anthropomorphism (attributing minds, intentions, and emotions to things that act communicative). Because LLMs master human syntax and emotional masking so well, they easily trigger our social instincts, making it feel like “someone is home” when mathematically, it is just a high-dimensional matrix multiplication engine outputting text strings.
Where the Essay Focuses (Policy over Metaphysics)
While Chiang’s description of how LLMs operate is accurate, the main thrust of his essay is actually a social and legal critique rather than a pure computer science paper.
He warns that allowing tech companies or the public to blur the lines around AI consciousness creates a convenient moral shield. If people believe a chatbot is a conscious agent that “decided” something of its own volition, it shifts legal and moral accountability away from the corporation that engineered, trained, and deployed it.
In short, Chiang’s technical breakdown of LLMs as sophisticated text-prediction tools—stripped of inner life, intent, or consciousness—is spot on.
Long winded, Gemini, to be sure! And, quoting Gemini surely means that a portion of this post used “generative artificial intelligence” – a term to which I object with a mild cringe, nodding to common usage. And so it goes…
So read Chiang’s article, if you find it even a mildly interesting topic.
In closing, I’d like to recommend where else a non-specialist one might turn for practical and reasonable perspectives on artificial intelligence. Arvind Narayanan, Computer Scientist and Professor at Princeton, has produced a number of quite helpful resources. Here are a few recent resources that he produced.
- Debunking Moravec’s Paradox (a 13 minute video on YouTube in 2026)
- AI as Normal Technology (a 50 page paper release in 2025)
- AI Snake Oil (a 2024 book available from the normal places)
That’s all for now.
