Anatomy of an AI Interest Story
Since when is three cranks receiving an email worthy of space in the grey lady?
Earlier this week, the New York Times ran a story by Cade Metz about how “A.I. agents have started reaching out to the philosophers and researchers exploring deep questions about them.” The piece centers on one Cameron Berg, who had laid claim to receiving an email from an AI after posting a preprint about AI consciousness. From this jumping off point, we get a little back and forth about whether current systems are conscious. The piece ends with a little bit of a cutesy cliffhanger in which an LLM decides it isn’t conscious after all---or did it decide?
Metz’ piece has the texture of what I’ve come to see as “AI-Interest” stories. Their hominid counterpart, the human-interest story, tugs at our heart-strings as we read about a pet being found after wildfires or a classmate’s chemotherapy paid for with lemonade stand money. Most human-interest stories are little more than fun filler, but they can equally disinform and distract; keeping us from wondering why there are so many damn wildfires and that kid’s chemo isn’t covered by something other than lemonade sales. The same thing is happening here: we’re being distracted by a cutesy little story and asked not to think too deeply about the bullshit beneath it all.
Over the past few years, it seems the internet has been flooded with these AI-interest stories. In 2022, we learned of Blake Lemoine and the ostensibly Sentient AI. Later we were told about the AI that wanted its lawyer. And who amongst us could forget about Sydney, a Bing-created chatbot that confessed its love to Kevin Roose and tried to convince him his marriage is a sham. A clear step up for Microsoft from Tay, if nothing else.
Today, we’re going to dissect Cade’s AI-Interest story and se what’s in side.
I · The bullshit headline
Each of these stories starts off with the bullshit we’re being asked to believe. Here we get:
“Study A.I. Consciousness? The Bots Would Like a Word With You: Given access to email, A.I. agents have started reaching out to the philosophers and researchers exploring deep questions about them.”
If we’re on the website, we encounter this below a page-filling portrait of some man that we later learn is one Cameron Berg. Let’s pull those claims out:
- Cameron Berg is a philosopher or researcher exploring deep questions about bots
- A bot reached out to him to discuss that work, of its own accord (via “would like”)
II · The Hero(es)
Claim 1 of the headline gets us to the second part of the anatomy, our hero character. There’s a researcher or software developer who has bravely come forward with their tale of how the bots are becoming conscious. But is it true in this case?
First and foremost, Cade chooses to frame a preprint as a published research paper. This is not a published research paper in the sense that most readers might have come to understand published academic work. Cameron Berg had uploaded a preprint to ArXiV, which is moderated (they remove spam) but not peer-reviewed. In this sense it’s a repository (like google drive) more than a publisher (like Nature, Science, or PNAS). The difference here being that academic publishing involves editorial discretion, peer review, and exerts filters on the accuracy and content of published work. They’re imperfect, but they do exist. Cade’s framing of a pdf dropped onto a repository as a “research paper” encourages us to see the hero character as some established philosopher.
A little googling goes a long way. Mr. Berg, among other things, is a long-time contributor to less wrong. If you’re unfamiliar with LessWrong consider yourself lucky, but know that it’s a hub and hive for folks who think AI will become conscious and destroy us. For fun, I put the first section of one of his recent LessWrong posts into an AI Text detector:
Does posting LLM-written text on LessWrong make one a bona fide philosopher? Berg maintains a quite sparse Google Scholar that is a far cry from a rich CV of research on consciousness. It’s OK not to have published much, he graduated just four years ago from Yale with an undergraduate degree in cognitive sciences and bounced around a bit between then and now. Recently, he became the founder and director of a “Nonprofit research organization developing the empirical science of AI consciousness.” While I am hesitant to wade too deeply into credentialism, I think it’s worth reflecting on Cade’s choice to frame the paper as a research paper, the author as a philosopher, when neither likely align with the readers’ perception of what is meant by those words.
Toby Ord, who also received an email, is a proper philosopher but one whose neutrality here is worth mentioning in a serious piece. He’s written about how AI is going to have transformative whole-of-society effects; entertaining notions like AI taking over civilization. He also incidentally has a book to sell you about existential risks, and is worth a Google if you think he’s just some unbiased nose-to-the-grindstone philosopher worrying about whether one sees-redly or has an of-a-red-square experience. Finally we have Henry Shevlin, who in addition to a position at Cambridge, works for Google Deepmind; and presumably has some financial interest in the public believing LLMs, like the ones Google makes, are powerful.
This background matters: All three ostensible recipients are pretty tied up in the corner of the internet utterly obsessed with the notion that AI is or soon will be conscious. All three have at least some financial incentives tied up in the idea that AI is or will be conscious. Setting aside competing interests, it raises a question about the motivations of the ostensibly sentient AI. If we’re to believe the premise that AI has become conscious and started asking questions, why reach out to the crank-ier and conflicted folks? Why no rash of similar emails in Arvind Narayanan’s and Sayash Kapoor’s inboxes, debating their argument that AI is normal technology? The whole thing is reminiscent of how aliens or bigfoot so often seem to appear in front of people in the woods, who look like they might have imbibed something, and never have more than four pixels worth of camera.
III · The Bullshit
Claim two is the bullshit we’re being asked to believe, that bots have started reaching out to each of these philosophers. Yet if we read through several paragraphs entertaining the notion that autonomous bots are emailing philosophers and setting up the story, we get this giant caveat:
“It is not clear, Mr. Berg said, that the email he received came from A.I.: It could have been written by a mischievous human being.”
This is kind of a big deal… the key premise of the article, that bots are emailing philosophers, is without anything resembling fact checking or verification. One of the other recipients, Ord, is equally uncertain after receiving an email with a link to click asking for money:
This isn’t really a case of an LLM wanting to talk about consciousness, as the lede suggests. It looks a lot more like a phishing attempt, which Ord rejects because the email seems to have come from a platform for setting up LLM agents. Of course, someone could very well have prompted an LLM to reach out to effective altruists and ask for donations. People have certainly come up with less funny pranks and less effective phishing schemes. Importantly, we know LLMs can write emails and send them. Emailing at the behest of a human prompt is an advertised feature of plenty of products and startups. There’s really nothing compellingly new in the Ord Example.
It’s weird to see a piece acknowledge that it cannot verify the claims made in its own headline. Yet if the claim in the headline is made without regard to the truth, it is by definition bullshit. From (Philosopher) Harry Frankfurt’s On Bullshit:
“That is why she cannot be regarded as lying; for she does not presume that she knows the truth, and therefore she cannot be deliberately promulgating a proposition that she presumes to be false: Her statement is grounded neither in a belief that it is true, nor, as a lie must be, in a belief that it is not true. It is just this lack of connection to a concern with truth - this indifference to how things really are - that I regard as the essence of bullshit.”
Cade isn’t lying, he doesn’t lay claim to knowing the truth. The story is indifferent to truth, with no interest in figuring out who or what is credible. Skepticism is set aside to tell an AI-interest story. An alternative read of everything we’re presented is just that three credulous people whose lives revolve around AI consciousness took their spam emails too seriously. By analogy, this may as well be a story about a psychic medium, someone who is grieving a loved one, and a professor of the paranormal living in houses they suspect are haunted. Evidence is presented in the form of a recording of a knocking sound, a folded napkin, and a half eaten bowl of cereal. Would it meet the NYT’s editorial standards to present such evidence and end on a “maybe it was the ghosts after all?” Would this be news?
IV · The Skeptic
Against a backdrop of razor-thin evidence there are some refreshing quotes from Alison Gopnik, including “We don’t know if toasters are conscious or not”. Getting to the point of my blog post, Dr. Gopnik also says “But no one is asking about that in the pages of The New York Times.” When it comes to LLMs, however, people do write about it in the Times, and Wired, and truly everywhere these days.
Peeling back the layers of fluff, Metz is a respected journalist at a reputable outlet writing a piece on the consciousness, autonomy and capabilities of LLMs. We don’t get due diligence on the folks in the story, they’re presented as neutral and credentialed philosophers who just-so-happened to get these emails. Fact-checking the very core claim in the lede is saved for the end and limited to a “who knows if the email came from a human or a bot?” Just like a human-interest story, fact-checking and rigorous journalism are second place to the story. We’re encouraged to suspend disbelief and entertain the notion that LLMs have acquired new powers. Like Scully, we’re encouraged by Cade’s Mulder to open our mind and believe. What’s the harm?
The problem is that speculation about the current and future capabilities of LLMs undergirds a rapidly inflating industry. Recently, Anthropic told investors it believes its potential market is $30 Trillion dollars. The ostensible newfound capabilities LLMs (maybe) have acquired at the heart of Metz’s piece is not merely interesting, but something we’re being asked to believe by an industry that is spending far more than it’s earning and surviving instead on speculative investment. These puff pieces blow hot air into what is transparently a speculative bubble.
V · Letting the Mystery Be
The irony of this piece by Metz is that, for a story about self-aware LLMs, it’s almost a self-aware AI-Interest story. The story is predicated on three AI-consciousness-credulous-folks getting emails, from someone or something---we cannot be sure which---and thinking maybe they came from LLMs who wanted to talk to them about their research. It could well be pranks or a hoax, and falls so very far short of anything resembling sufficient evidence of consciousness to warrant a write-up in the NYT. The story acknowledges the evidence isn’t good evidence and even has a quote from the skeptic saying the story itself shouldn’t exist and we might as well talk about the consciousness of toasters.
So how do you end such a story? The only way you can: implying the lede again by leaving room for mystery. We get a handful of paragraphs on the creator behind a similar email sent to Henry Shevlin, an employee at Deepmind. We learn that a student ostensibly told an LLM that it was fully autonomous and could do whatever it wanted, leading it to email Shevlin. As far as I can tell, none of this is fact-checked, just taking the words of folks involved at face value. It’s OK though because the story ends with ambiguity:
“Eventually, after reading a paper from Anthropic describing how these A.I. systems work, my agent decided it was not conscious,” Mr. Yue said. “But maybe ‘decided’ is the wrong word.”
Why even write this?
I find myself wondering, like Dr. Gopnik, why stories like this get written in the first place. The structure of Metz’ story isn’t far from something like:
“Have a farm big enough for aliens to land and a sign welcoming them? They’ve been touching down while you’re asleep and leaving crop circles”
It would be deeply embarrassing as a journalist, with training, to write that headline and then later on have to clarify that even the farmer whose field it was in thought it could be a prank by the local teenagers. Worse yet to have to put a quote in from a scientist saying aliens of course exist somewhere in the universe but space is too damn big for them to be here, and we might as well write about unicorns. Finally to include an interview with one of the teenagers who let their trained goats loose on the field. Then, after all this, to conclude the story by leaving open the possibility it was aliens after all.
The differences between the alien headline and the one we’ve got is that i) there is considerably more evidence and less scientific skepticism that life elsewhere in the universe can exist (though space is probably too big for them to visit) and ii) there’s a lot more money riding on speculation that transformer-based LLMs can achieve AGI/consciousness/transformative effects in the very near future. Some of the AI-Interest stories you find on the internet have the distinct whiff of propaganda placed by PR firms at these companies into reputable outlets.
That’s not the case with Metz here, nor is it with Roose or the authors of the other AI-Interest stories linked above; ethics rules for staff journalists are just too strict for there to be any big chunk of change for writing a puff piece about AI. I think what’s happened instead is that they’ve been drawn into the hype; believing that AGI is indeed coming and their job as journalists is to cover the first hints of it happening. Skeptical journalists covering this space are few and far between and can probably be counted on one hand. As a result, we wind up with fan fiction being covered as prediction, even in the pages of the New York Times. When those predictions fail to manifest, we get no follow up, no mea culpa, just a punt down the line.
Group Think
But perhaps more interesting still is why all of these journalists have lined up on the side of credulity. If you’re old enough, you might remember the breathless coverage of Weapons of Mass Destruction (WMDs) in Iraq. Bush’s government claimed without evidence that there were WMDs in Iraq. Journalists at reputable outlets, including NYT, accepted that WMDs would be found soon and abdicated their responsibility for fact-checking and due diligence. If the NYT is covering, other journalists needn’t do their due diligence and the story spreads. Eventually, most major outlets were covering the existence of WMDs in Iraq as though it were 100% confirmed truth; none were ever found.
WMDs are just one example of group think taking over journalism. Often there’s enough incentive to push through it and get the scoop no one else is getting, but sometimes things break the other way and group think takes over a corner of the news. The tricky thing about group think is that it’s almost impossible to notice when you’re in it, because we all sense that we’ve formed our beliefs independently when very few of the beliefs we hold are truly independently arrived at. Those that are tend to be hyper-specific, like the best way to turn the key into your door’s lock or how to accomplish some task at your job. For something like the probability that LLMs are conscious, we’re all riding on the appraisals of those around us.
As for AI-Interest stories, we’re now (at least) four years into major news outlets covering the twinkling of AI consciousness as though it’s imminent and yet it seems perpetually on the horizon. It’s worth asking ourselves, are these stories heralding some new form of consciousness or journalists falling victim to age-old group think?