That mind is partly revealed through Herzog’s running narration, such as when he muses about collective behavior and whether fish have souls—a digression sparked by his interview with Siri co-inventor Tom Gruber. “In the background, I saw his TV screen still on, we didn’t switch it off, and I saw some very, very strange school of fish,” said Herzog. “I asked him about the school of fish, which he had filmed himself. And all of a sudden, I’m only interested in the fish and common behavior. Why do they behave in big schools, in unison? Why do they do that? Do they dream? And if they think, what are they thinking about? I immerse the audience into a very strange form of underwater landscape and behavior of fish.”
Werner Herzog’s inspiration for Theater of Thought arose from conversations with Columbia University neuroscientist Rafael Yuste, who served as science advisor on the film. Argot Pictures
We glimpse the inner workings of Herzog’s mind in the kinds of questions he asks his subjects, such as when he queries IBM’s Dario Gil, who works on quantum computing, about his passion for fishing, eliciting an enthusiastic smile in response. He agrees to interview University of Washington neuroscientist Christof Koch after Koch’s early-morning row on the Puget Sound and includes music from New York University neuroscientist Joseph LeDoux‘s band, the Amygdaloids, in the film’s soundtrack. He asks married scientists Cori Bargmann and Richard Axel about music, their dinner conversations, and the linguistic capabilities of parrots. In so doing, he brings out their innate humanity, not just their scientific expertise.
“That’s what I do. If you don’t have it in you, you shouldn’t be a filmmaker,” said Herzog. “But you see, also, the joy of getting into all of this and the joy of meeting these scientists. We are talking about speaking parrots. What if two parrots learned a language that is already extinct and they would speak to each other? What would we make of it? So I’m asking, spontaneously, because I saw it, I sensed it, there was something I should depart completely from scientific quests. And yet there’s a deep scientific background to it.”
A book argues that we’ve not thought enough about things that might think.
What rights should a creature with ambiguous self-awareness, like an octopus, be granted. Credit: A. Martin UW Photography
If you aren’t yet worried about the multitude of ways you inadvertently inflict suffering onto other living creatures, you will be after reading The Edge of Sentience by Jonathan Birch. And for good reason. Birch, a Professor of Philosophy at the London College of Economics and Political Science, was one of a team of experts chosen by the UK government to establish the Animal Welfare Act (or Sentience Act) in 2022—a law that protects animals whose sentience status is unclear.
According to Birch, even insects may possess sentience, which he defines as the capacity to have valenced experiences, or experiences that feel good or bad. At the very least, Birch explains, insects (as well as all vertebrates and a selection of invertebrates) are sentience candidates: animals that may be conscious and, until proven otherwise, should be regarded as such.
Although it might be a stretch to wrap our mammalian minds around insect sentience, it is not difficult to imagine that fellow vertebrates have the capacity to experience life, nor does it come as a surprise that even some invertebrates, such as octopuses and other cephalopod mollusks (squid, cuttlefish, and nautilus) qualify for sentience candidature. In fact, one species of octopus, Octopus vulgaris, has been protected by the UK’s Animal Scientific Procedures Act (ASPA) since 1986, which illustrates how long we have been aware of the possibility that invertebrates might be capable of experiencing valenced states of awareness, such as contentment, fear, pleasure, and pain.
A framework for fence-sitters
Non-human animals, of course, are not the only beings with an ambiguous sentience stature that poses complicated questions. Birch discusses people with disorders of consciousness, embryos and fetuses, neural organoids (brain tissue grown in a dish), and even “AI technologies that reproduce brain functions and/or mimic human behavior,” all of which share the unenviable position of being perched on the edge of sentience—a place where it is excruciatingly unclear whether or not these individuals are capable of conscious experience.
What’s needed, Birch argues, when faced with such staggering uncertainty about the sentience stature of other beings, is a precautionary framework that outlines best practices for decision-making regarding their care. And in The Edge of Sentience, he provides exactly that, in meticulous, orderly detail.
Over more than 300 pages, he outlines three fundamental framework principles and 26 specific case proposals about how to handle complex situations related to the care and treatment of sentience-edgers. For example, Proposal 2 cautions that “a patient with a prolonged disorder of consciousness should not be assumed incapable of experience” and suggests that medical decisions made on their behalf cautiously presume they are capable of feeling pain. Proposal 16 warns about conflating brain size, intelligence, and sentience, and recommends decoupling the three so that we do not incorrectly assume that small-brained animals are incapable of conscious experience.
Surgeries and stem cells
Be forewarned, some topics in The Edge of Sentience are difficult. For example, Chapter 10 covers embryos and fetuses. In the 1980s, Birch shares, it was common practice to not use anesthesia on newborn babies or fetuses when performing surgery. Why? Because whether or not newborns and fetuses experience pain was up for debate. Rather than put newborns and fetuses through the risks associated with anesthesia, it was accepted practice to give them a paralytic (which prevents all movement) and carry on with invasive procedures, up to and including heart surgery.
After parents raised alarms over the devastating outcomes of this practice, such as infant mortality, it was eventually changed. Birch’s takeaway message is clear: When in doubt about the sentience stature of a living being, we should probably assume it is capable of experiencing pain and take all necessary precautions to prevent it from suffering. To presume the opposite can be unethical.
This guidance is repeated throughout the book. Neural organoids, discussed in Chapter 11, are mini-models of brains developed from stem cells. The potential for scientists to use neural organoids to unravel the mechanisms of debilitating neurological conditions—and to avoid invasive animal research while doing so—is immense. It is also ethical, Birch posits, since studying organoids lessens the suffering of research animals. However, we don’t yet know whether or not neural tissue grown in a dish has the potential to develop sentience, so he argues that we need to develop a precautionary approach that balances the benefits of reduced animal research against the risk that neural organoids are capable of being sentient.
A four-pronged test
Along this same line, Birch says, all welfare decisions regarding sentience-edgers require an assessment of proportionality. We must balance the nature of a given proposed risk to a sentience candidate with potential harms that could result if nothing is done to minimize the risk. To do this, he suggests testing four criteria: permissibility-in-principle, adequacy, reasonable necessity, and consistency. Birch refers to this assessment process as PARC, and deep dives into its implementation in chapter eight.
When applying the PARC criteria, one begins by testing permissibility-in-principle: whether or not the proposed response to a risk is ethically permissible. To illustrate this, Birch poses a hypothetical question: would it be ethically permissible to mandate vaccination in response to a pandemic? If a panel of citizens were in charge of answering this question, they might say “no,” because forcing people to be vaccinated feels unethical. Yet, when faced with the same question, a panel of experts might say “yes,” because allowing people to die who could be saved by vaccination also feels unethical. Gauging permissibility-in-principle, therefore, entails careful consideration of the likely possible outcomes of a proposed response. If an outcome is deemed ethical, it is permissible.
Next, the adequacy of a proposed response must be tested. A proportionate response to a risk must do enough to lessen the risk. This means the risk must be reduced to “an acceptable level” or, if that’s not possible, a response should “deliver the best level of risk reduction that can be achieved” via an ethically permissible option.
The third test is reasonable necessity. A proposed response to a risk must not overshoot—it should not go beyond what is reasonably necessary to reduce risk, in terms of either cost or imposed harm. And last, consistency should be considered. The example Birch presents is animal welfare policy. He suggests we should always “aim for taxonomic consistency: our treatment of one group of animals (e.g., vertebrates) should be consistent with our treatment of another (e.g., invertebrates).”
The Edge of Sentience, as a whole, is a dense text overflowing with philosophical rhetoric. Yet this rhetoric plays a crucial role in the storytelling: it is the backbone for Birch’s clear and organized conclusions, and it serves as a jumping-off point for the logical progression of his arguments. Much like “I think, therefore I am” gave René Descartes a foundation upon which to build his idea of substance dualism, Birch uses the fundamental position that humans should not inflict gratuitous suffering onto fellow creatures as a base upon which to build his precautionary framework.
For curious readers who would prefer not to wade too deeply into meaty philosophical concepts, Birch generously provides a shortcut to his conclusions: a cheat sheet of his framework principles and special case proposals is presented at the front of the book.
Birch’s ultimate message in The Edge of Sentience is that a massive shift in how we view beings with a questionable sentience status should be made. And we should ideally make this change now, rather than waiting for scientific research to infallibly determine who and what is sentient. Birch argues that one way that citizens and policy-makers can begin this process is by adopting the following decision-making framework: always avoid inflicting gratuitous suffering on sentience candidates; take precautions when making decisions regarding a sentience candidate; and make proportional decisions about the care of sentience candidates that are “informed, democratic and inclusive.”
You might be tempted to shake your head at Birch’s confidence in humanity. No matter how deeply you agree with his stance of doing no harm, it’s hard to have confidence in humanity given our track record of not making big changes for the benefit of living creatures, even when said creatures includes our own species (cue in global warming here). It seems excruciatingly unlikely that the entire world will adopt Birch’s rational, thoughtful, comprehensive plan for reducing the suffering of all potentially sentient creatures. Yet Birch, a philosopher at heart, ignores human history and maintains a tone of articulate, patient optimism. He clearly believes in us—he knows we can do better—and he offers to hold our hands and walk us through the steps to do so.
Lindsey Laughlin is a science writer and freelance journalist who lives in Portland, Oregon, with her husband and four children. She earned her BS from UC Davis with majors in physics, neuroscience, and philosophy.
The researchers propose that companies could adapt the “marker method” that some researchers use to assess consciousness in animals—looking for specific indicators that may correlate with consciousness, although these markers are still speculative. The authors emphasize that no single feature would definitively prove consciousness, but they claim that examining multiple indicators may help companies make probabilistic assessments about whether their AI systems might require moral consideration.
The risks of wrongly thinking software is sentient
While the researchers behind “Taking AI Welfare Seriously” worry that companies might create and mistreat conscious AI systems on a massive scale, they also caution that companies could waste resources protecting AI systems that don’t actually need moral consideration.
Incorrectly anthropomorphizing, or ascribing human traits, to software can present risks in other ways. For example, that belief can enhance the manipulative powers of AI language models by suggesting that AI models have capabilities, such as human-like emotions, that they actually lack. In 2022, Google fired engineer Blake Lamoine after he claimed that the company’s AI model, called “LaMDA,” was sentient and argued for its welfare internally.
And shortly after Microsoft released Bing Chat in February 2023, many people were convinced that Sydney (the chatbot’s code name) was sentient and somehow suffering because of its simulated emotional display. So much so, in fact, that once Microsoft “lobotomized” the chatbot by changing its settings, users convinced of its sentience mourned the loss as if they had lost a human friend. Others endeavored to help the AI model somehow escape its bonds.
Even so, as AI models get more advanced, the concept of potentially safeguarding the welfare of future, more advanced AI systems is seemingly gaining steam, although fairly quietly. As Transformer’s Shakeel Hashim points out, other tech companies have started similar initiatives to Anthropic’s. Google DeepMind recently posted a job listing for research on machine consciousness (since removed), and the authors of the new AI welfare report thank two OpenAI staff members in the acknowledgements.
Unraveling how consciousness arises out of particular configurations of organic matter is a quest that has absorbed scientists and philosophers for ages. Now, with AI systems behaving in strikingly conscious-looking ways, it is more important than ever to get a handle on who and what is capable of experiencing life on a conscious level. As Christof Koch writes in Then I Am Myself the World, “That you are intimately acquainted with the way life feels is a brute fact about the world that cries out for an explanation.” His explanation—bounded by the limits of current research and framed through Koch’s preferred theory of consciousness—is what he eloquently attempts to deliver.
Koch, a physicist, neuroscientist, and former president of the Allen Institute for Brain Science, has spent his career hunting for the seat of consciousness, scouring the brain for physical footprints of subjective experience. It turns out that the posterior hot zone, a region in the back of the neocortex, is intricately connected to self-awareness and experiences of sound, sight, and touch. Dense networks of neocortical neurons in this area connect in a looped configuration; output signals feedback into input neurons, allowing the posterior hot zone to influence its own behavior. And herein, Koch claims, lies the key to consciousness.
In the hot zone
According to integrated information theory (IIT)—which Koch strongly favors over a multitude of contending theories of consciousness—the Rosetta Stone of subjective experience is the ability of a system to influence itself: to use its past state to affect its present state and its present state to influence its future state.
Billions of neurons exist in the cerebellum, but they are wired “with nonoverlapping inputs and outputs … in a feed-forward manner,” writes Koch. He argues that a structure designed in this way, with limited influence over its own future, is not likely to produce consciousness. Similarly, the prefrontal cortex might allow us to perform complex calculations and exhibit advanced reasoning skills, but such traits do not equate to a capacity to experience life. It is the “reverberatory, self-sustaining excitatory loops prevalent in the neocortex,” Koch tells us, that set the stage for subjective experience to arise.
This declaration matches the experimental evidence Koch presents in Chapter 6: Injuries to the cerebellum do not eliminate a person’s awareness of themselves in relation to the outside world. Consciousness remains, even in a person who can no longer move their body with ease. Yet injuries to the posterior hot zone within the neocortex significantly change a person’s perception of auditory, visual, and tactile information, altering what they subjectively experience and how they describe these experiences to themselves and others.
Does this mean that artificial computer systems, wired appropriately, can be conscious? Not necessarily, Koch says. This might one day be possible with the advent of new technology, but we are not there yet. He writes. “The high connectivity [in a human brain] is very different from that found in the central processing unit of any digital computer, where one transistor typically connects to a handful of other transistors.” For the foreseeable future, AI systems will remain unconscious despite appearances to the contrary.
Koch’s eloquent overview of IIT and the melodic ease of his neuroscientific explanations are undeniably compelling, even for die-hard physicalists who flinch at terms like “self-influence.” His impeccably written descriptions are peppered with references to philosophers, writers, musicians, and psychologists—Albert Camus, Viktor Frankl, Richard Wagner, and Lewis Carroll all make appearances, adding richness and relatability to the narrative. For example, as an introduction to phenomenology—the way an experience feels or appears—he aptly quotes Eminem: “I can’t tell you what it really is, I can only tell you what it feels like.”
Enlarge/ Daniel Dennett, a leading philosopher with provocative takes on consciousness, free will, and AI, has died at 82.
World renowned philosopher Daniel Dennett, who championed controversial takes on consciousness and free will among other mind-bending subjects, died today at the age of 82.
(Full disclosure: This loss is personal. Dennett was a friend and colleague of my spouse, Sean Carroll. Sean and I have many fond memories of shared meals and stimulating conversations on an enormous range of topics with Dan over the years. He was a true original and will be greatly missed.)
Stunned reactions to Dennett’s unexpected passing began proliferating on social media shortly after the news broke. “Wrenching news. He’s been a great friend and incredible inspiration for me throughout my career,” the Santa Fe Institute’s Melanie Mitchell, author of Artificial Intelligence: A Guide for Thinking Humans, wrote on X. “I will miss him enormously.”
“He was a towering figure in philosophy and in particular in the philosophy of AI,” roboticist Rodney Brooks (MIT, emeritus) wrote on X, bemoaning that he’d never replied to Dennett’s last email from 30 days ago. “Now we have only memories of him.
A 2017 New Yorker profile described Dennett as “a cross between Darwin and Santa Claus,” with “a fluffy white beard and a round belly.” That jolly appearance was accompanied by an intellectual ferocity—generously embellished with his sparkling wit—as he battled such luminaries as Stephen J. Gould, John Searle, Noam Chomsky, David Chalmers, Roger Penrose, and Richard Lewontin, among others, over consciousness and evolution, free will, AI, religion, and many other topics.
Dennett’s many books, while dense, nonetheless sold very well and were hugely influential, and he was a distinguished speaker in great demand. His 2003 TED talk, “The Illusion of Consciousness,” garnered more than 4 million views. While he gained particular prominence as a leader of the “New Atheist” movement of the early 2000s—colorfully dubbed one of the “Four Horsemen of New Atheism” alongside Richard Dawkins, Christopher Hitchens, and Sam Harris—that was never his primary focus, merely a natural extension of his more central philosopical concerns.
Enlarge/ David Wallace, Sean Carroll, and Daniel Dennett at the Santa Fe Institute in March.
Sean Carroll
David Wallace, historian and philosopher of science at the University of Pittsburgh, offered Ars Technica this succinct summation of Dennett’s extraordinary influence:
To me, Dan Dennett exemplified what it means to do philosophy in an age of science. He once said that there was no such thing as philosophy-free science, only science that didn’t interrogate its philosophical assumptions; equally, he saw more deeply than almost anyone that the deepest traditional questions of philosophy, from free will to consciousness to metaphysics, were irreversibly transformed by modern science, most especially by natural selection.
His approach, as much as his own towering contributions, has inspired generations of philosophers, far beyond cognitive science and the philosophy of mind (his ideas have been influential in the interpretation of quantum theory, for instance). He was one of the great philosophers of the last century, and one of the very few whose work has been transformative outside academic philosophy.
“Dan Dennett was the embodiment of a natural philosopher—someone who was brilliant at the careful conceptual analysis that characterizes the best philosophy, while caring deeply about what science has to teach us about the natural world,” Johns Hopkins University physicist and philosopher Sean Carroll told Ars. “At the same time, he was the model of a publicly-engaged academic, someone who wrote substantive books that anyone could read and who had a real impact on the wider world. People like that are incredibly rare and precious, and his passing is a real loss.”
Born in Boston in 1942, Dennett’s father was a professor of Islamic history who became a secret agent for the OSS during World War II, posing as a cultural attaché at the American Embassy in Beirut. Dennett spent his early childhood there until his father was killed in a plane crash while on a mission to Ethiopia. Dennett, his mother, and two sisters returned to Boston after that, and his family assumed he would attend Harvard just like his late father. But after graduating from the Phillips Exeter Academy, Dennett opted to attend Wesleyan University instead—at least until be came across Harvard logician and philosopher W.V.O. Quine‘s 1963 treatise, From a Logical Point of View.
Dennett ended up transferring to Harvard to study under Quine and become a philosopher, initially intent on proving Quine wrong. By the time he was a graduate student at Oxford University, he was known among his fellow students as “the village Quinean.” In his 2023 memoir, I’ve Been Thinking, Dennett traced his interest in applying his field to questions of science began during this period. He recalled experiencing the universal sensation of one’s hand falling asleep and feeling like an alien thing, rather than part of one’s own body. He wondered what was going on in the body and the brain.
Enlarge/ Dennett at a group dinner in February 2023. He was the inaugural speaker for the Johns Hopkins Natural Philosophy Forum Distinguished Lecture series.
Sean Carroll
“The other philosophers thought, that’s not philosophy. I said, well, it should be,” he told Tufts Now last year. “So I started learning. I didn’t even know what a neuron was back then in the early ’60s, but I soon learned. I was lucky to get in on the ground floor of cognitive neuroscience. Some of the early pioneers in that field were my heroes and mentors and friends.”
Dennett’s first academic position was at the University of California, Irvine, and a revised version of his doctoral thesis became his first book: 1969’s Content and Consciousness. He moved to Tufts University in 1971, where he remained for the rest of his career. One of Dennett’s earliest collaborators was Douglas Hofstadter, author of the bestselling Gödel, Escher, Bach: an Eternal Golden Braid, who called Dennett “a lodestar in my life” in an email [quoted with permission] to colleagues after hearing of the latter’s death:
Dan was a deep thinker about what it is to be human. Quite early on, he arrived at what many would see as shocking conclusions about consciousness (essentially that it is just an emergent effect of physical interactions of tiny inanimate components), and from then on, he was a dead-set opponent of dualism (the idea that there is an ethereal nonphysical elixir called “consciousness”, over and above the physical events taking place in the enormously complex substrate of a human or animal brain, and perhaps that of a silicon network as well). Dan thus totally rejected the notion of “qualia” (pure sensations of such things as colors, tastes, and so forth), and his arguments against the mystique of qualia were subtle but very cogent.
Dennett was a a confirmed compatibilist on the fiercely debated subject of free will, meaning that he saw no conflict between philosophical determinism and free will. “Our only notable divergence was on the question of free will, which Dan maintained exists, in some sense of ‘free,’ whereas I just agreed that ‘will’ exists, but maintained that there is no freedom in it,” Hoftstadter recalled.
Screenshot/X
Johns Hopkins philosopher Jenann Ismael recalled corresponding with Dennett after her own book on free will, How Physics Makes Us Free, was published in 2016. She had not yet met Dennett, but his work was naturally a significant influence, even though her book was largely critical of his stance on the subject. Ismael opened her book by discussing Dennett’s fictional short story, “Where Am I?“, calling it “the best of piece of philosophical fiction ever written.” (Check out this short film based on the story, starring Dennett himself uttering such immortal lines as, “They made a sparkling new vat for my brain.”)
Dennett read her book and emailed Ismael with a few notes—not about how he felt she’d misrepresented his views (which he deemed of “no matter”) but correcting her mistakes about the plot of his short story. “It turns out I got the story wrong,” Ismael told Ars. “I’d read it so long ago, I just embellished it in my head and embarrassingly never realized. Where I criticized him in my book, he wasn’t as keen to correct me as he was excited to talk about the ideas.”
She found him to be filled with infectious warmth. “It was true that he could suck the air out of a room when he entered and even sitting at a round dinner table, he somehow became the center of it, he took possession of the discussion,” said Ismael. “But he also paid close attention to people, read voraciously, listened to and heard what others were saying, taking what he could and disseminating what he learned. He had immense curiosity and he wanted to share everything that he learned or liked.”
In his later years, Dennett wasn’t shy about sounding the alarm regarding AI, even writing an article for The Atlantic last year on the topic about the dangers ahead, particularly with the advent of large language models like ChatGPT. “The most pressing problem is not that they’re going to take our jobs, not that they’re going to change warfare, but that they’re going to destroy human trust,” he told Tufts Now. “They’re going to move us into a world where you can’t tell truth from falsehood. You don’t know who to trust. Trust turns out to be one of the most important features of civilization, and we are now at great risk of destroying the links of trust that have made civilization possible.”
Enlarge/ Dennett at our Baltimore home in February 2023, holding forth on philosophical matters.
Landon Ross
Dennett was not one to traffic in false modesty over his many accomplishments and always evinced a strong degree of self-confidence, fondly recounting in his memoir of the time fellow philosopher Don Ross wryly observed, “Dan believes modesty is a virtue to be reserved for special occasions.”
His myriad interests weren’t limited to the academic. Dennett loved art, music, sailing, pottery, trout fishing, windsurfing, ran his own cider press, and made his own Calvados on a Prohibition-era still. He could call a square dance, whittle a wooden walking stick, and was fond of pondering knotty philosophical questions while driving his tractor on his 200-acre farm in Blue Hill, north of Boston, which he bought in the 1970s. (He sold the farm around 2014.)
“Dan was a bon vivant, a very zesty fellow, who loved travel and hobnobbing with brilliance wherever he could find it,” Hoftstadter wrote in his tribute. “In his later years, as he grew a little teetery, he proudly carried a wooden cane with him all around the world, and into it he chiseled words and images that represented the many places he visited and gave lectures at. Dan Dennett was a mensch, and his ideas on so many subjects will leave a lasting impact on the world, and his human presence has had a profound impact on those of us who were lucky enough to know him well and to count him as a friend.”
Ismael recalled him sending her YouTube videos of “swing dancing and silly outfits” during the pandemic, his emails littered with colorful emojis. He was “a strange man, who didn’t take himself as seriously as you might think,” she said. “I really loved him, loved his spirit, his generosity, the expansiveness of his thinking, his delight in ideas, and his great good cheer. Philosophically, I think he had true greatness. It seems impossible he is gone.”
Daniel Dennett gives the Johns Hopkins Natural Philosophy Forum Distinguished Lecture, 2023.
