It’s like this: Why perceptions are our realities
Minds do not create experience, experience creates minds
Cecilia Bleasdale just needed a dress to wear to her daughter’s wedding. She had no idea what was about to happen. After considering several frocks, she sent photos of three to her daughter, Grace, informing her she’d purchased one of them.
“Oh, the white and gold one?” Grace replied.
“No,” Cecilia answered. “It’s blue and black.”
Try as they might, mother and daughter couldn’t agree on how the gown appeared in the blurry smartphone photo. Within days, millions of people around the world were having the exact same argument. Some saw “The Dress” as white and gold. Others, with equal confidence, saw it as blue and black. Numerous celebrities weighed in as well. Taylor Swift, who saw the dress as blue and black, said the controversy made her “confused and scared.”
It’s easy to see why. While we all share the same planet, each of us really does live in our own little world—something we don’t often notice. If you’re a political person, you might get a feel for this when viewing social media posts from people who disagree, but The Dress meme provided something more—a microcosm example of how individualized perception that can often be uncontrollable.
That’s because our brains mostly function beneath the surface of our awareness. And for good reason: Imagine the cognitive overload of always needing to tell your lungs to breathe or your heart to pump. But we don’t even have full control over our voluntary senses either. If you saw The Dress as gold and white, you couldn’t simply will your neurons to see it as black and blue. The Dress wasn’t just a silly internet debate—it was a gateway to the realization that minds do not create experience but rather that experience creates minds.
Humans often like to think we’re far superior to other animals, but science keeps casting doubt on that idea. Chimpanzees and other primates are famous for their intelligence, but cetaceans are also extremely smart; some have even been documented using an equivalent to personal names that marine biologists call signature whistles. Crows, octopuses, elephants, and parrots have also shown incredible abilities to use tools, play, and remember things years later.
What truly sets us apart from other rational animals is our ability to adapt to change, and to create it ourselves. When faced with habitat destruction or violent threats from rivals or predators, ancient humans moved, often hundreds of miles away. This is why Homo sapiens has the biggest range of any large animal in the world, and why we have become the dominant species on the planet despite being significantly weaker, smaller, and slower than many others.
Our brains are our greatest biological asset, which is more than a little ironic considering how little we understand them. Neuroscience offers many insights into how brains work and how mental internality arises from within the nervous system, but it leaves many questions unanswered. That’s why a new discipline of studying minds—cognitive science—has emerged, bundling ideas from neuroscience with those from psychology, ethology (animal behavior), philosophy of mind, and machine learning.
There are no perfect answers as of this writing, but there is so much to think about.
ELIZA’s revenge
Breaking academic silos can be difficult, especially amid recent budget cuts in some countries. A positive trend for cognitive-science funding has been the emergence of large language model (LLM) artificial intelligence organizations, which have attracted tens of billions of dollars from investors after OpenAI’s ChatGPT launched in late 2022 and drew 100 million users in just two months, smashing the previous record set by TikTok in nine months.
Based on a new “transformer” architecture, ChatGPT was hailed as a technical marvel, able to answer difficult questions, solve logic puzzles, and hold conversations in a way that struck many people as eerily human. The underlying GPT-3 model was seen as a vindication of OpenAI researchers’ claim that ramping up data sources and computing power would exponentially increase model accuracy.
“We observe no signs of deviation from these trends on the upper end, though performance must flatten out eventually,” they asserted in a January 2020 paper that previewed their methodological approach.
ChatGPT was an undeniable success. Besides inspiring dozens of imitators and attracting hundreds of millions of users, it easily passed the so-called Turing Test of being able to converse with humans without being identified as a computer runtime state. The goal of AI engineers since the days of the first chatbot, ELIZA, had been accomplished.
But is being able to converse with a human all that it takes to be conscious? Are people talking about the same things when they use the word consciousness? Is throwing more computing time at the algorithms the path to super-intelligent machines? Not everyone was convinced that scale was all you needed to get to a human-like mind. While GPT-3 and other chatbots proved capable of a wide variety of other tasks (including writing software, analyzing documents, and fielding customer-service inquiries), as AI industry gadfly Gary Marcus and many others have documented extensively, LLMs frequently make things up, generate non-functional code, fail at basic math, and image-generating diffusion models routinely mangle scenes.
That was supposed to change in August 2025 with GPT-5, according to OpenAI CEO Sam Altman.
“With GPT-5 now, it’s like talking to an expert—a legitimate PhD-level expert in anything, any area you need, on demand. They can help you with whatever your goals are,” he said in a preview video.
But many ChatGPT users disagreed once the model launched, posting on social media almost immediately that they considered the new bot to be worse than previous models. The backlash was so intense that OpenAI was forced to restore the old GPT-4o version to placate its angry paying customers. The company’s reputation tumbled as well. Before GPT-5, bettors at Polymarket gave OpenAI a 35 percent chance of having the best model by the end of 2025, a number that fell to 17 percent after the new chatbot debuted.
While OpenAI has faced the most recent torrent of criticism, it isn’t accurate or fair to single out the company. Every transformer-based LLM has struggled with the same types of problems since the architecture debuted in 2017. Despite all the computing power, training time, and ingested data, the neural network model of breaking inputs into “tokens” and then mapping their statistical relationships has not consistently yielded correct results. This is particularly evident in image generators, which routinely fail to understand user instructions to place objects in the correct relation to one another, as Marcus and other critics frequently point out.
Eventually, even Altman had to publicly admit that his detractors had a point: “This is clearly a model that is generally intelligent, although I think in the way that most of us define AGI, we’re still missing something quite important, or many things quite important,” he said at a news conference.
Color me intrigued
To get a better idea of what AI is missing, let’s return to The Dress. Cecilia Bleasdale and everyone who saw it in person reported it as black and royal blue. Yet when you use a software color-picking tool on Bleasdale’s photo, the gown registers as gold and light grayish blue. So what gives?
It’s a fascinating question, one that boils down to this: color is a perceived quality rather than an inherent one like mass. Many people cannot distinguish red from green or blue from yellow, but even if they could, there is no single definition of any color. Beyond informal cultural conventions, dozens of national and international organizations publish color standards.
Take red, for instance. According to the sRGB color gamut that’s the web standard, the red primary color is defined as 100% red, 0% green, and 0% blue. And sure enough, when you view a square of that hue on a computer monitor, it appears unmistakably red.
But is it “pure” red? Not if you ask Digital Cinema Initiatives, whose P3 color space can define that same exact sRGB hue as 92% red, 20% green, and 14% blue.
How is that possible? Because color gamuts differ in size and are designed to meet the differing needs of the organizations that define them. In the ink-focused CMYK color space, there is no red primary at all. Things get even stranger when you consider that the red primary of the Display P3 standard looks no different from sRGB red on a traditional computer monitor, but on a newer HDR monitor, the color appears brighter and more brilliant than what sRGB insists is pure red. (You can see for yourself how these comparisons work if you have an HDR monitor.)
So, is P3 red the reddest of them all? No. Since there are other color space standards that encompass more of the visible light spectrum, this means there are reds that are redder still. Things get trickier still once you realize that the term “visible light” is itself a separate standard based on experiments conducted in 1931 with only 17 subjects. Given that visual acuity degrades over lifespans, there are likely thousands, if not millions, of children who can see a redder red than the visible light standard.
There’s something about Mary’s room
The abstract concept of red is an agreed-upon convention to facilitate communication, but what is the basis for that agreement? Is it purely arbitrary, or is there something more fundamental underneath? It’s a question that is often considered via the philosopher Frank Jackson’s famous “Mary’s Room” thought experiment. In the scenario, a woman named Mary has somehow been trapped within a chamber in which color does not exist. She and everything within it are black and white. Coincidentally, however, Mary is a color scientist who knows all the abstract facts about color. There is literally nothing about how color is perceived by humans that she does not know.
One day, a package arrives at her door. Inside is a color television. When she turns it on, does Mary learn something?
To some observers, Jackson’s scenario, often called the knowledge argument, is proof of an underlying aspect of reality, something that Mary could only access after seeing color for the first time. But for Jackson, she had acquired something “epiphenomenal,” a quale, a mental anchor of knowledge that doesn’t exist in the physical world:
“It seems just obvious that she will learn something about the world and our visual experience of it,” he writes. “But then it is inescapable that her previous knowledge was incomplete. But she had all the physical information. Ergo there is more to have than that, and physicalism is false.”
Although Jackson intended to describe a non-religious epistemic construct, presupposing a layer beneath physical reality also aligns with theistic dualism, in which an unseen universe or force is both the origin of the physical universe and the epistemic grounding point of mental internality. According to dualists, people “know what it’s like” to experience something because it’s based on something metaphysical, be it the Platonic-Christian logos, Kant’s noumenal realm, or the handiwork of sundry creator gods.
The epiphenomenalist argument offers a certain elegance (and at least some reassurance to those who wish to believe in deities). But it also presents a fundamental problem: If qualia are the epiphenomenal basis of the world while having no actual interaction or influence over it, on what basis could they be proven to exist? How can they ground experience if their existence is non-evidentiary? Belief in their necessary existence seems to be a mere deferral of infinite regress.
These and other problems with epiphenomenalism eventually led Jackson to join what he later called the “physicalists.” Mary did not learn any new information, he now says, rather she gained an ability to describe something she had already known. That was fellow philosopher Daniel Dennett’s take as well. Over a decades-long career, he relentlessly attacked the scenario as hopelessly unrealistic because it asks readers to believe the impossible premise that a person could know literally everything physical about color perception. A self-proclaimed “computational functionalist,” Dennett argued that qualia are nothing more than illusions, the products of desperation to retain the “Cartesian theater” of dualism.
It’s hard to disagree that the Mary’s Room thought experiment is problematic. But while it’s true that Mary wouldn’t learn any new abstract facts upon viewing color for the first time, it seems rather apparent that turning on the color television would provide her with something: what color looks like to her, an internal somatic reference point she’d never had before.
While the linguistic imprecision of the Mary’s Room scenario easily lends itself to false resolutions, the question raised by its proper interpretation is worth consideration, especially since it abuts the “hard problem of consciousness,” a term coined by philosopher David Chalmers to describe what he considers a fundamental dilemma for physicalist theories of minds:
Why is it that when our cognitive systems engage in visual and auditory information-processing, we have visual or auditory experience: the quality of deep blue, the sensation of middle C? How can we explain why there is something it is like to entertain a mental image, or to experience an emotion? It is widely agreed that experience arises from a physical basis, but we have no good explanation of why and how it so arises. Why should physical processing give rise to a rich inner life at all? It seems objectively unreasonable that it should, and yet it does.
Trying to resolve this dilemma, Chalmers has offered a concept he and others call “panpsychism,” an elaborate and yet rather vague argument that the universe itself is made of experiential entities—be they atoms, quantum systems, or something else—and that these tiny minds are the stuff from which larger minds, and their experiences, are made.
The argument resonates in a certain way with many religions, but ultimately, it’s an intellectual dead-end. Chalmers often seems to acknowledge as much, frequently stating that panpsychism is just an idea he’s throwing out, even admitting regularly that its claims seem not to be conducive to scientific experimentation. Likewise, Chalmers also concedes that his argument is nonspecific about which objects have consciousness. Sometimes, large inanimate objects like mountains do, sometimes they don’t. How one could measure or perceive inorganic consciousness also seems impossible to tell. And as with epiphenomenalism, if consciousness is everywhere but doesn’t seem to do anything measurable, then this belief provides no useful information even if it’s true.
While very interesting, panpsychism and dualism don’t seem to be coherent, but nonetheless, the concept of experience is worth exploring, especially since as Thomas Nagel (whose famous essay, “What Is It Like to Be a Bat?” elevated the topic of phenomenal experience in philosophy) has written elsewhere, making room for analysis of mental phenomena is no threat to physical science:
“To say that there is more to reality than physics can account for is not a piece of mysticism: it is an acknowledgment that we are nowhere near a theory of everything, and that science will have to expand to accommodate facts of a kind fundamentally different from those that physics is designed to explain.”
What are minds?
One possible solution to these issues is what neuroscientist Anil Seth calls “biological naturalism,” the view that phenomenal experience requires a living basis or substrate, an idea that he has developed from the arguments of the philosopher John Searle. This position has intuitive appeal: every entity we know that seems to have feelings or sentience is alive, after all.
In his famous “Chinese Room” thought experiment, Searle sought to demonstrate that computation alone cannot generate understanding. Here’s how the scenario works:
Imagine yourself locked in a room with an instruction manual for manipulating Chinese characters. People slide questions written in Chinese under the door, you follow the manual’s rules to compose responses, and slide them back out. From outside, it appears you understand Chinese—but you’re merely shuffling symbols according to syntactic rules, with no grasp of what any of it means.
From this fact, Searle concluded that syntax (formal symbol manipulation) can never yield semantics (meaning), and therefore mindedness cannot be purely computational. Something about biology must be the source of genuine understanding.
In his book Being You and in essays for Noema and other magazines, Seth builds on Searle’s foundation by incorporating Predictive Processing theory—the idea that brains constantly generate models of sensory inputs and update these models based on prediction errors. In this framework, perception is not a passive reception of sensory data but an active inference process. As Seth vividly puts it, phenomenal experience is a “controlled hallucination,” the brain’s beliefs about what’s causing its sensory signals, continuously calibrated against actual inputs.
The brain’s primary job, according to Seth, is keeping the body alive, and the experiences of emotion, mood, and embodied selfhood arise from perceptual inferences about bodily condition, which derive from perceptual predictions involved in controlling and regulating bodily condition. These processes are deeply rooted in our nature as biological, self-regenerating systems resisting entropic decay. The “feeling of being alive” isn’t some abstract phenomenon, it’s the lived model of a biological system maintaining itself against thermodynamic dissolution. Brains are not computers, he argues in his Noema essay, and reductionists like Dennett who insist that cognition is nothing but computation are getting it wrong:
Abstracting the brain into the arid sequence space of algorithms does justice neither to our biology nor to the phenomenology of the stream of consciousness. […]
When we see the brain for what it really is, the notion that all its multiscale biological activity is simply implementation infrastructure for some abstract algorithmic acrobatics seems rather naı̈ve. The brain is not a Turing machine made of meat.
But here’s the problem: Predictive Processing is itself fundamentally computational, formalized through Bayesian inference, prediction error minimization, and hierarchical generative models—all algorithmic processes that can be (and routinely are) implemented digitally. This becomes significant when we consider that developmental biologist Michael Levin has recently been utilizing Predictive Processing to argue for precisely the kind of substance dualism Seth wants to avoid. In his “Ingressing Minds” paper, Levin proposes that ideas and anatomical patterns are Platonic forms that intrude into physical matter, and that bodies and even computer algorithms are “interfaces” for these eternal entities. According to Levin, biological development works because organisms use their “agential material” to minimize error in pursuit of patterns that exist in an unknown dimension.
The fact that the same theoretical framework—Predictive Processing—can be marshaled to support both Seth’s materialist monism and Platonic dualism should give us pause. Either Levin has erred horribly in his application of Predictive Processing, or the theory is metaphysically neutral and biological naturalism requires something else to support its assertion that phenomenal experience cannot be implemented computationally.
If experience and meaning arise from prediction error minimization and Bayesian inference (purely digitizeable processes), then Seth’s entire argument against computational functionalism evaporates. If they arise from some special property of living matter, then he owes us an explanation of what that property is.
At best, it seems that biological naturalism recreates the very epiphenomenal qualia that Seth and other physicalists are trying to avoid. His “controlled hallucinations” become mental models that are somehow separate from cellular functions yet also controlling them—exactly the kind of non-physical causation that epiphenomenalism proposed and that physicalists have rightfully rejected.
All that said, however, Seth is correct about a very crucial point: Minds are not separable from the substrates that make them. As he rightly notes, you cannot replace a biological neuron with a silicon equivalent while leaving its function perfectly preserved, because neurons do things like clearing metabolic waste that silicon cannot replicate. The materiality of the substrate matters. But Seth misses the full implication of his own insight by clinging to inadequate computational metaphors. Minds are neither software nor hardware—they are execution states.
In computing, an execution state is the complete configuration of a running process at a particular moment in time. It’s not the program code (software) that defines what could happen, nor the physical circuits (hardware) that make computation possible. It’s the active, temporally unfolding activity that determines what’s happening now. An execution state includes register values, memory contents, instruction pointers, stack frames—the entire dynamic configuration that makes the running process what it is at that instant.
Execution states have several critical properties that map perfectly onto experiencing minds:
They are temporal. An execution state exists only while the process is running. If the process stops, the mind stops—even if the body and its idea-contents are still there. Death and near-death experiences illustrate this well. Mind-as-process also mirrors Seth’s own observation that phenomenal experience is richly dynamic and inherently temporal, flowing rather than stuttering from state to state. It also mirrors how most large language models work in practice. Their internalities are inactive unless they are being trained or responding to user input.
They are causally efficacious. The current execution state constrains which states can follow. You cannot arbitrarily jump to any possible future state—the next state must follow causally from the current configuration. This is mental causation without invoking non-physical forces.
They are substrate-dependent but not substrate-identical. This is the crucial distinction Seth overlooks. A functional organization can be described independently of its physical realization without implying independence from physical realization. A sorting algorithm can be described abstractly, but every actual execution of that algorithm requires specific hardware. Similarly, we can describe patterns of neural activity without being committed to the view that these patterns could float free from neural tissue.
In biological cognition, mental internality is the coordinated activity pattern across cellular and tissue-level subagents: neuronal firing regimes, neurotransmitter dynamics, bio-electric fields, hormonal states, metabolic constraints, immune signaling, and tissue-level coordination. It is not reducible to anatomy—the same anatomical structure can enact radically different execution states. A brain in deep sleep versus active problem-solving has the same anatomy but vastly different execution states. Yet execution states are not independent of anatomy either. No substrate, no enactment.
A metaphor from computer gaming illustrates this distinction clearly. The Pokémon video game franchises often feature nearly identical software with only minor modifications—different character sprites and names, but otherwise identical game logic and algorithms. It may seem at first that these small differences do not matter, but in fact they do. When you try to load an archived execution state (a “save file”) from Pokémon Let’s Go Eevee using Pokémon Let’s Go Pikachu, incompatibilities emerge. Items don’t match their proper positions, character data becomes corrupted, and the game may behave erratically—even though the underlying hardware remains totally unchanged.
You don’t have to take my word for this either. My 14-year-old daughter pointed me toward a video by YouTuber “PokeTips Mike” which illustrates all three relationships at once: the execution state is distinct from both the software (the game code) and the hardware (the console), yet it depends on the specific interaction between them and the player.
No two execution states will ever be the same, even on the same device running the same software. And you can forget about trying to load a Let’s Go save file on an older Nintendo Wii system or on a totally different game like Pokémon Unite.
For biological minds, the execution state is the collaboration of cellular and tissue collectives. It’s not Cartesian software running on “wetware” bodies. It’s not reducible to the mere presence of neurons and synapses. It’s the temporally extended, causally constrained, substrate-dependent activity that those neurons and synapses are currently engaged in—an activity that includes but is not limited to the computational processes Seth describes.
This idea (which is part of a larger theory I will be writing and speaking about more in the near future) preserves everything valuable in biological naturalism while avoiding its contradictions. Life matters because biological substrates support particular kinds of execution states that silicon currently cannot. The multiscale integration Seth emphasizes, the autopoiesis of self-producing cellular machinery, the continuous engagement with thermodynamic time—all of these create execution states with properties that purely digital computation lacks.
But this is an empirical claim about current technology, not a metaphysical claim about computational functionalism. If future technologies could replicate the relevant substrate properties—not just the abstract information processing, but the material dynamics of living systems—then they might support similar execution states and thus similar forms of mindedness. The question shifts from “Is experience computational?” to “What material properties must a substrate have to support the execution states that constitute experience?”
This dissolves the apparent contradiction in Seth’s position. Predictive Processing can be part of the story without being the whole story. Minds are not the hardware, not the software, but the ongoing collaboration of the body’s components. Seth is right that biology is integral to how cognition works, but his framework is incomplete.
Perception and externality
I don’t pretend to have all the answers, but if we work from a starting point of minds as execution states, I think it gets us a lot further to better understanding the hard problem of consciousness. Biology is essential to how our minds work, but it is not essential to how minds must work.
Each of us exists within externality, the space that we observe outside of mental internality, but our understanding of the former is fundamentally limited. Individually, we occupy unique slices of spacetime and are too large to see atoms with our eyes, too slow to outrun a greyhound, and too short-lived to survive a dash to Sirius.
But beyond our collective limitations, we also have individual ones. Some of us have difficulty hearing higher-pitched sounds due to ear damage. And some of us (like me) cannot perceive that The Dress as blue and black because our brains over-correct the lighting in the store where Cecilia Bleasdale was shopping, interpreting it as a daytime ambiance rather than a blue-dominant nighttime setting.
Our experience of externality is integral to what we think it is. As the philosopher Maurice Merleau-Ponty cautioned in Phenomenology of Perception: “We must therefore avoid saying that our body is in space, or in time. It inhabits space and time.”
Perception is not reality, but it is the only reality that we can individually know—our percepted externality.
Psychologists offer many accounts of how perception and reasoning work, but a widely influential one came from Nobel laureate Daniel Kahneman, a psychologist and economist who popularized Dual Process Theory, a framework arguing that reasoning generally can be broken down into two types: an instinctive, embodied mode which he called System 1, and a more effortful, recursive mode he called System 2.
Not all reasoning is like this, of course, and internality is not just two mental modes arguing constantly. Still, decades of research suggest that most of the time we use our embodied somatic reasoning (my term for System 1) to navigate the world, but sometimes we use abstract reasoning (my term for System 2) when we encounter more difficult obstacles.
Ethology research has found that similar cognitive dynamics exist within animal minds. Many species demonstrate abstract reasoning capacities—including recognizing themselves, colors, shapes, materials, scents, and number concepts. Multiple primate species have been able to learn sign languages. Orcas have been repeatedly observed teaching hunting techniques to their young. Alex, an African gray parrot, became famous for being able to correctly use human language concepts, including asking what color he was, and even instructing other parrots in English.
Kahneman often framed the two reasoning systems as competitors, with the abstract correcting and overriding the somatic when necessary. However, neuroscientists like Antonio Damasio have shown that cognition always begins with sensation, echoing the insight of David Hume that deliberation is the servant of the passions. In other words, abstract reasoning is a tool of somatic reasoning, not its rival. Intuition isn’t corrected by logic; it allows itself to be persuaded—or not, as in the case of debates over The Dress or which city has the best pizza.
Embodied cognition research also indicates this as well. As George Lakoff, Mark Johnson, and others in cognitive linguistics argue, all abstract concepts are rooted in conceptual metaphors that derive from past experiences. Nothing is ever perceived alone. “This” is always “like that.”
Context shapes experience. Having to listen to a group of six-year-olds play violin together is surely banned by at least one Geneva Convention, yet their parents often gladly pay hundreds or even thousands of dollars for the privilege. Similarly, hearing the jingle of an ice cream truck evokes memories of childhood bliss for many Americans, but it summons no such sensations for residents of China or Japan.
There’s no doubt that grounding the abstract within the somatic is useful, but it does not answer the hard problem of consciousness. Why do things feel “like that?” To answer, we must go even deeper.
Pointing toward meaning
The limits of perception are fundamental to how we experience externality, and as The Dress episode demonstrated, we all live in our own little slice of it. Over time, humans have expanded what we can perceive through inventions such as telescopes, microscopes, radar, and simulated infrared vision. All of this has been made possible through an even greater invention—language—the social tool that enables us to compare our limited private observations with those of others, continuously expanding the collective percepted externality.
Anthropologists and linguists lack a firm consensus on when language evolved, but the most recent estimate holds that it became common about 100,000 years ago. Although scholars disagree on timelines, they generally agree that one of the very few lexical constructs shared by the approximately 7,000 known natural languages is the presence of “shifter” words, whose meaning is context-dependent. These words change meaning according to the speaker’s desired frame of reference—from this to that, here to there, now to later. This act of shifting attentional focus to a shared referent is called deixis.
Infant-cognition research indicates that deixis is fundamental to language acquisition, but the idea of identifying referents appears to be even more basic and somatic. Many animals direct attention with indicative gestures, even some of the putatively most simple.
The “waggle dance” of honeybees is prelinguistic deixis. When a forager who has found a new food source returns home, it will wiggle-walk at an angle relative to the top of the hive to indicate where the object is relative to the sun. This is remarkably advanced for a nonverbal creature since the solar azimuth is a neutral reference point that works for any observing bee, regardless of which direction it may be looking to watch. But the cognitive achievement doesn’t stop there: the further away the food source is, the longer the dancer will make its procession. Some will even waggle more prominently to indicate food quality.
Bees may be the most sophisticated deictic communicators among insects, but they are far from alone. Ants and termites use a variety of differentiated methods to establish referents, including pheromones, chirps, and scraping body parts to create sounds. Many species use sound to indicate their own position during mating season to potential partners. Nonlinguistic deixis exists in smaller organisms as well—and for good reason: whether you’re a professor or a protist, focusing attention on a stimulus is the beginning of responding to it.
Besides Predictive Processing, there are many other theories about how complex systems (agents) perform cognition. Global Workspace Theory, Higher Order Thought theories, and Recurrent Processing are among the most influential. For our purposes, however, we will focus on how cognition begins, which these theories generally tend to bracket.
While humans are many things, at the most basic level, we are multicellular organisms—group projects of trillions of unthinking eukaryotic parts that perform different functions in a cooperative manner in concentric systems of complexity. Larger agents such as animals are made of organ systems (pulmonary, nervous, digestive, etc.) which are themselves made of organs and connective structures like vessels or tracts. These structures are composed of specialized tissues, which are in turn made of individual cells. At every level of complexity, each piece of the whole is separated from similar subagents and the larger system of which it is part.
Biologists who study the evolution of multicellularity find that it has generally evolved in two major ways: either multiple organisms come together to form a larger one (“aggregative”) or a single cell divides into new ones that remain together (“clonal”). In either case, multicellular organisms have significant advantages over unicellular ones, since they can move faster, travel further, live longer, and consume smaller ones.
But being multicellular presents its own problems of coordination and cooperation. But coordinating responses across huge numbers of cells is a difficult feat, and it necessarily begins with a two-step process that can be called somatic deixis.
The first step, designation, seems like a simple act, and functionally it is, which is why even unicellular agents like amoebas manage it. Upon encountering a stimulus, an organism’s individual sensorimotor cells or organelles (its subagents) send their percepted observations about it to a local processing center (such as a cellular nucleus or the larger tissue system of which they are part), confirming that something is being observed. Each cell’s output is severely limited, but the collective combination yields something much more significant.
The compound eyes of insects provide a useful comparison of how this works: each individual ommatidium cannot see, but when combined with tens of thousands of others, a complete visual picture emerges—one that is oftentimes more complex than what humans can perceive.1
Cognition is correlation. Whether the stimulus appears to exist outside the agent and is perceived through exteroception, or the stimulus is inside and perceived through interoception, does not matter. Because the cells generating the information have direct access to the stimulus, their reports are treated as functionally true. Any entity that constantly doubts its own sensory data would not survive.
The incredible thing about biological entities is that the knowledge their cell collectives create scales upward as they live and grow. Research from the biologist Pamela Lyon and others indicates strongly that all clonal multicellular eukaryotes use precognitive forms of electrochemical communication to perceive externality and coordinate organism-level responses to it. This is how somatic deixis begins—even without neurons.
Once a stimulus is anchored as a referent within percepted externality (“this is here/there”), simple cognitive agents stop deictic evaluation and move straight to responding. When a portion of a slime mold detects a food source, the other portions move toward it. When a bacterium detects a harmful acid, it reverses course. When a plant’s photoreceptors detect sunlight, it orients its leaves toward it.
In more complex agents, after designation has been completed, the referent is placed into context through the second step of somatic deixis—adjudication—a prelinguistic evaluation in which assessments of the stimulus are shared among relevant subagents, some of which have access to stored memories of past experiences with similar referents, including the referent’s qualities and actions, the agent’s actions, and the agent’s past interoceptive states.
Once the adjudication process is complete, the referent is anchored within internality and has a “what this feels like” significance. With the two deictic steps completed, the agent has produced a somatic token that it can use immediately for thought or action, or store for future reference. The token has been confirmed by multiple, distinct subagents, and also has an immediate emotional component derived from the agent’s past and present bodily responses to the stimulus.
Somatic tokens are qualia. But rather than being the mysterious product of cognition, they are its primal basis.
Scaling from the somatic
The ability to create somatic tokens is a very clear evolutionary advantage. Being able to remember past stimuli and one’s own responses to them is exceptionally useful. Across food webs, more cognitively complex organisms dominate simpler ones.
But the recursive nature of cognition does not stop with the formation of somatic tokens. Over time, organisms living in more cognitively demanding ecological niches began processing their somatic tokens counter-factually. Instead of asking only “do what with this?” they began asking “what will this do?” Animals that perceived others as separate cognitive agents whose choices could be predicted had a tremendous survival advantage (especially predators). But it was more than that. Having a theory of mind about other agents increased the development of their own minds.
Being able to place stimuli not just within a single context but across contexts is what abstract reasoning does. It’s a tool to extend and correct somatic perceptions through counterfactual logic that can continuously recur on itself to create its own abstract tokens, which are conceptual and symbolic, adding a “what this is about” layer to the original somatic core.
Working together, both types of reasoning can reevaluate each other’s concepts to generate other learned behaviors, including sentience (awareness of others and awareness of more internal states), selfhood (the awareness that “I” am distinct from “Not-I” and the creation of small constructed realities, inter-related collections of tokens), and eventually consciousness—the ability to create complex constructed realities like languages, arts, sciences, and mathematics, all of which can be used to study and modify internality and externality in an endless variety of ways.2
As much of an advantage as cognitive correlation is for agents, however, it is a lossy process. Because cognition is a distributed, continuous workflow created by unintelligent cellular subagents, their basic outputs are extremely difficult to examine at higher epistemic levels.
In biological agents, there are two separate explanatory gaps that make cognition difficult to analyze. The first is vertical and inside the agent. As physical and cognitive abstraction increase, awareness of how sensation, significance, and context are created is lost, because the somatic tokens—rather than the primal chemical and physical interactions of the cells—are transmitted upward in the cognitive chain. While an agent can fully believe and act upon its somatic tokens, because of how they are created, organism-level abstract reasoning cannot completely understand deictic experience. This means that no biological agent will be able to perfectly explain to itself what a sensation feels like.
An analogy from computing is instructive here. Inside every modern cell phone there is a chip called an accelerometer that uses a tiny “proof mass” that moves in response to external acceleration. Another chip, called a gyroscope, uses a tiny vibrating structure that measures how fast the device is turning (if at all).
If I’m a software developer creating a game that requires users to tilt their phones to move a character, I don’t need to know how accelerometers or gyroscopes work. I don’t even need to know they exist. That’s because software programs called “drivers” convert the devices’ electrical signals into data that are fed to the phone’s central processor, and then combined by its operating system into rotation variables which I can access via a software layer called the application programming interface (API).
Cognitive abstraction is power, but its price is an internal epistemic gap. Humans trying to fully comprehend their own subjective experiences are like software developers trying to directly inspect the electromagnetic readings of an accelerometer. In a cellphone, the hardware and software abstraction layers that make the rotation data useful also make the underlying physics inaccessible. In a biological system, the cells that directly experience externality cannot be directly accessed by the internality that they jointly create.
But the vertical epistemic gap is not the only obstacle to understanding the richness of experience. There’s a horizontal one as well. Because no agent can fully explain its own somatic tokens using the abstract tokens of language, it will also be unable to explain them to others. This is why people can sincerely claim to be “at a loss for words.” In that moment, their abstract reasoning is unable to accurately encode their somatic experience into abstract language. While their ideas, feelings, and beliefs are completely believed, they are unable to be fully conceptualized.
But even if it were possible to perfectly encapsulate one’s qualia into language, the deictic references on which they are based are incommunicable. Somatic tokens created to describe this referent for this agent at this moment make less sense the further removed they are from the original experience. Thomas Nagel’s argument that it’s impossible for a human to know what it’s like to be a bat is correct, for the same reason that it’s impossible for any other human to fully know what it’s like to be Thomas Nagel.
This dilemma, often called the problem of other minds in philosophy, is why communication is often so difficult. That being said, however, agents can still share reports of their experience with each other, and agents that have inhabited sufficiently similar percepted externalities can find overlapping experiences to achieve cooperation, even without complete understanding. This is the task of language, science, rational discourse, and even the arts—to continually expand the collective percepted externality.
Pointing toward intentional computing?
As we’ve seen, the shrouded nature of felt experience is not a metaphysical mystery, it’s an ontological necessity. Still, deictic focus can be imperfect, as the world discovered during The Dress controversy. Psychologists have demonstrated this as well through numerous experiments, including the famous “rubber hand illusion,” in which test subjects demonstrate reflexive identification with a rubber hand after one of their actual hands is covered and stroked in synchrony with the fake one.
Externality exists regardless of our opinions about it, but our subjectivity is the only way we can access it. Private subjectivity is the necessary grounding for public veridicality. The possibility that The Dress can be known as actually blue and black depends on it being able to be mis-perceived as white and gold. Being able to be wrong about the world is the only possible foundation for being right about it. Understood correctly, the longstanding conflict between scientific realism and constructivist perspectivism dissolves.
Where does this leave us with respect to artificial intelligence? After several years of immense hype, the idea that scaling alone is the sole route to an intentional AI system has collapsed. Newer models purporting to offer “chain-of-thought reasoning” have been revealed to be only simulating it. A March 2025 survey of AI industry leaders found that 76 percent believed that scaling up current AI approaches was unlikely to lead to an “artificial general intelligence” system.
Some AI theorists have proposed that generative models use somatic reasoning because they respond quickly to inputs, but this misunderstands Dual Process Theory, which actually holds that System 1/somatic reasoning is embodied and emotion-laden. Instead of using somatic reasoning, current AI systems are abstract token processors whose lack of deictic grounding contributes significantly to their output failures. Despite their remarkable ability to break down training data into abstract tokens and reassemble it into responses, LLMs and other generative models are ultimately just shuffling ungrounded symbols.
While somatic deixis appears simple for biological agents (most animals seem capable of it), it is more complex ontologically than it seems. Invertebrate honeybees can reorient within the hive, but image generators often struggle with outputting rotated representations. Being able to mark a stimulus as within the entity (interoceptive) or outside the entity (exteroceptive) seems to be an abductive act that no amount of deductive logic can reliably duplicate.
As researchers like Anthropic’s Amanda Askell and many others have demonstrated, alignment rules and reinforcement training from human operators do often improve model outputs, but these methods are primarily about modifying or prohibiting outputs rather than constraining internal computational processes. Unfortunately, alignment strictures are merely guardrails rather than integrated somatic reasoning.
We’re still far removed from a truly intentional AI system, even though current models will surely continue to improve around the margins. But this does not necessarily mean that it can never happen. Today’s generative models already seem to mirror the idea of cognitive subagents through their “attention heads” that select relevant tokens and combine them into sequentially larger structures to ultimately form outputs. Embodied robots can navigate spaces and perform a rudimentary form of somatic designation on objects as they navigate around lab environments. The likely path to intentional AI is to combine the two inside of a larger system that also matches what we’ve seen in biology:
Continuous state persistence
Multiscale integration
Autopoiesis/self-maintenance
Thermodynamic embedding
Interoception and exteroception integration
That’s much easier said than done, of course, but if somatic deixis is the basis by which unthinking, connected biological subagents build cognition, perhaps it can also provide a cognitive scaffolding for unthinking artificial subagents. Internality is emphatically not what Gilbert Ryle derided as a “ghost in the machine;” it is a ghost of the machine. If an intentional AI is ever to be invented, it will need to be this as well. Selves are not imbued; they are grown. Experience creates minds.
Somatic deixis is a new concept, as far as I can tell, but it’s eminently eminently testable, particularly in volvocine algae and in baker’s yeast species, which let us compare unicellular states with clonal multicellularity (volvocines) and aggregative versus clonally developing group formation (yeast). Research on stimulus response and adaptation across these forms already exists, but there is substantial room for more targeted experimental designs.
Having a precise definition of consciousness allows us to better study and understand it. Far too often, the word is used as a floating signifier that oscillates between experience, cognition, sentience, meta-cognition, and self-alignment.