The conclusion to me, is that we may be able to build systems that behave intelligently before we understand whether intelligence and consciousness are connected at all.
For a non-bat to experience what it is like to be a bat, you have to embrace one of two philosophies:
- Dualism: body and soul/consciousness are separate), or
- Panpsychism: consciousness is fundamental and doesn't emerge from the material physiology.
For a materialist, and someone who thinks consciousness arises from the physical aspects, the idea of a human experiencing bat consciousness is not possible. Our evolution developed algorithm for processing the world is wired to our senses. Similarly a bat's perception of the world has evolved along with bat senses and is not the same as ours.
Without any of the evolutionary pre-wiring, a human conscious dropped into a bat would be deaf, dumb and blind.
> I assume we all believe that bats have experience.
Humorously enough, earlier he refers to those who believe that non-human mammals are not all conscious people as "extremists", so it's clear he understands this is not a fully accurate assumption.
Two separate meanings of "have experience" are being swapped interchangeably, I think: one is "brain can sense the world around the entity, react to changes, and act or plan actions", and one is all that plus "implements a person, or point of view, or subjectively aware entity that supervises experiencing", which is to say, a person. What it is like to be a bat could be rephrased as what it would be like to experience being a bat if a person were being a bat, but that doesn't actually imply that bats implement or contain a personal point of view. If they don't, then it might be that there is no "what it is like to be a bat", but at most "what it is like to experience being a bat as a person implemented by a system which is not a bat".
I think he makes it pretty clear he's only talking about the second one of your two definitions
>What it is like to be a bat could be rephrased as what it would be like to experience being a bat if a person were being a bat
He says:
>[what] it would be like for me to behave as a bat behaves. But that is not the question. I want to know what it is like for a bat to be a bat
The point is that bats do have a subjective experience of the world which is very different from a person's. It seems like you think only humans have this?
I have always liked the way that this paper frames the distinction and tension between the feeling of subjective experience and the "detached" rational scientific descriptive perspective that purports to be outside of that experience.
What is Real by Adam Becker was a fun foray into why this is so in (some) modern science philosophy as well - there's some desire to say that there isn't a "there" there when we talk about the world, just stuff. I'm probably with Alan Watts on the whole thing, that we are in some sense local aspects of a larger consciousness pretending it isn't so, and the hard work done by detached, disembodied perspectives like the scientific descriptive one are more and more steps to an unfolding game.
I read this article since it was referred to often in philosophy of mind, including by Daniel Dennett in "Consciousness Explained".
Yet... while I expected some deeper dive into Umwelts, I got (in my experience) a tautology around the word "be". Which, IMHO, should be tabooed in all serious philosophical discussion, as "be" is the mother of word-lockpicks. Vide E-Prime, English without "be", https://en.wikipedia.org/wiki/E-Prime.
Assuming you are planning to continue participating anyway, you should just respond. Presuming it's a good response, with a bit of luck someone will vouch for it and make it visible.
I think we have a pretty good explanation today - it's like embeddings from AI models. Experience is both content and reference, we represent new experience in relation to old experience. That makes representation personal, being made of one's own past experience. This does not explain away pure feeling, but explains how we make discriminations of similarity and difference between our experiences, the contents of qualia, the qualitative aspects.
We also know brains are locked inside a bone box only connected to the outside world by a bundle of unlabeled nerves, there is no direct access. So the brain can only compare patterns of signals it receives from outside. But since this representation-action-learning loop is recursive it cannot be inhabited or known from outside, 3p needs to pay the price of recursion to execute in order to get to 1p.
The gap is that between description and execution, which cannot be crossed for free with cheap description. Execution costs, and that cost is part of what is like being a bat. We can't inhabit their cost pressures since we don't have their context and body. You can't remove the costs of being a bat from "what it is like being a bat" and still get your answer from the comfort of the philosophical armchair.
You nailed it. Asking the question is asking to define from the outside what is an inner recursive process. The question is a simple confusion of domains. This is Humbert Maturana’s main point in Autopoiesis and Cognition (1980, now reissued). Recommend the whole book, as does Terry Winograd. The most intense part is the appendix specifically about the nervous system. Nagel and others knew no neuroscience and are clueless about recursion.
Isn't Maturana's theory that consciousness has to do with language, and the use of language to make distinctions about ourselves and others? To me, this seems clearly insufficient to explain consciousness - qualia totally precede language; one could experience qualia without language, etc.
Random thought I had on bats since they "see" by hearing reflected sounds:
Can bats know what another bat is looking at or even see what another is seeing by listening to the other's echoes? I imagine they can also recognize each other's voices and so identify individuals in flocks with the images they are seeing. I imagine this would be like being able to beam a stream of visual information into another's head.
I think the answer to your first question is mostly yes, because we know that when traveling in large swarms, many bats go quiet so they don't overwhelm the signal, yet they still manage to navigate fine.
What it's like - the gestalt of a bat (or other thing) as it engages its sensing-deciding-reacting loop. This gestalt isn't just for biological organisms, but any system for which its decision making engages with representations of the external environment unified with a self-representation to form a coherent representation of a persistent entity engaged with an external world.
Why do such systems need this gestalt? Why consciousness instead of everything happening in the dark? The recognition of oneself as situated in the world is crucial to coherent engagement with the world. It is how an entity can ensure its body parts are moving towards the same goal. It's how behavior over time doesn't undermine its purpose. Fragmented, incoherent behavior does not serve self-preservation.
LLMs as they are currently constructed probably aren't conscious, but we are a hop skip and a jump away from ones that are.
> The recognition of oneself as situated in the world is crucial to coherent engagement with the world. It is how an entity can ensure its body parts are moving towards the same goal. It's how behavior over time doesn't undermine its purpose. Fragmented, incoherent behavior does not serve self-preservation.
Why would movement towards a goal be incoherent if it happened "in the dark"? Our brains perform many critical functions "in the dark" (and do so coherently) which do not rise to the level of consciousness.
Presumably the question you're asking is why does a unified self representation require consciousness. (Split brain cases are easy examples of how a break in unification results in incoherent behavior.) The brain nominally performs functions as cascading behavior of atoms whose structural relationships correspond to various functions. But there is no unification at the unconscious/atomistic level therefore a new representational regime is required that can ground the higher level unification.
An organism must exhibit a high level of competence at reacting appropriately to environmental/sensory states. The "light's being on" is how the brain represents being situated in a world and the significant features therein. Representations within this gestalt are inherently meaningful. For example, phenomenal pain brings with it competence at protecting bodily integrity. The memory of pain becomes part of the explanatory narrative for the monitoring function that tracks progress towards goals ensuring coherent behavior (imagine being fearful of a stove but not knowing why). The contents of consciousness is the semantic engine that induces competent behavior over time on otherwise naive entities.
I agree that evolution could not produce a rational agent who would still reliably respond to lower level imperatives (such as pain, hunger, lust) without consciousness and feeling. The primitive parts of the brain have to be able to override the higher functions to ensure survival and reproduction. But an LLM isn't evolved in this way; its fitted to a functional output. It is entirely possible there will never be anyone home. I sure hope there isn't, because at the scale we're using them it would be a moral catastrophe.
I don't think that it is appropriate to use "gestalt" here. The word used in the field is "qualia", it has a precise meaning and is precisely what Nagel was writing about. Gestalt, to my understanding, is quite different, even when used in english psychology writing.
My usage of gestalt isn't without precedent[1]. I like gestalt better than qualia as a neutral description of the explanandum. Qualia is an atomistic view of consciousness and so is heavily theory-laden. I had just read a comment from the previous thread[2] on how this paper was translated into other languages and the lack of an equivalent "what its like" phrasing. The translations struck me as missing the virtue of the what it's like phrasing, namely identifying the intrinsic perspectivalness of cognitive systems without taking a stand on how to cash it out. I was trying to think of a better phrasing that could translate well and I landed on gestalt.
> This gestalt isn't just for biological organisms, but any system for which its decision making engages with representations of the external environment unified with a self-representation to form a coherent representation of a persistent entity engaged with an external world.
This doesn't seem quite right, or at least underspecified. We can talk about this stuff concretely these days, at least in the context of digital systems. E.g. i can draw up a diagram of a system that takes in some camera and audio data (and tactile, proprioceptive, etc.), tokenizes it then runs that + past state data through some autoregressive VLM to drive an inference process. The state being passed around can be written out analytically for a given trained model - the external and internal environmental representations, the linear algebra that transforms them into latent action representations, the process by which that is transformed into control signals. It seems difficult to claim that the computational process that implements this has any more or less of a gestalt then one multiplying two matrices together. So it's not just the existence of certain representations or computational loops that seems to lead to possessing a gestalt.
> It seems difficult to claim that the computational process that implements this has any more or less of a gestalt then one multiplying two matrices together. So it's not just the existence of certain representations or computational loops that seems to lead to possessing a gestalt.
I've thought a lot about what is lacking in modern VLMs that preclude consciousness. In my view the difference is that their talk of "self" is a simulacrum of the real thing. Current models are feed forward and so self-talk is driven by some parameter that turns on when the network detects context that possibly references the model, and this parameter drives downstream self-talk. It's a very good simulacrum, but it is a far cry from a model with recurrent self-reference around which the inference process is organized. The richness of the self-model in a hypothetical recurrent network with capabilities of modern LMs is much greater than the parameter on/off representation in feed forward networks.
Completely agree. This is what Hofstadter means by a strange loop. Our current LLMs have no attentional autonomy by design. The recursion is superficial and without its own Now. Adding attentional autonomy is The frightening alignment issue.
Seems like a rather ad hoc restriction. The issue is one of inferring the structure of the processes generating the output. I suppose given enough time and an adversarial style of interaction one could in principle determine the computational structure of any system with high confidence. So probably yes, modulo real-world concerns.
I've read this paper several times, over the years, and I've never understood why its considered such a landmark. I think it could be losslessly compressed to one paragraph of modest length.
Many people (e.g., Daniel Dennett, myself) think it's a terrible paper that has done a huge amount of damage. Unfortunately, that too makes it a landmark. Likewise for Searle's Chinese Room argument and Chalmers' "hard problem" as discussed in "The Conscious Mind" and elsewhere.
An Octopus may seem more like an alien with its 9 brains (one central, plus one per tentacle), but note that the left/right hemispheres of other animal's brains are essentially separate brains, and human's who have had "split brain" surgery to separate these (e.g. in cases of severe epilepsy) don't report feeling much, if at all, different afterwards than they did before.
I would expect that an Octopus's central brain may well feel as if it is directly controlling it's arms, and receiving sensory feedback from them, even though it is not.
The reality is that we don't see the external world - we predict it (and receive error feedback), and similarly our brain can't also help but predict itself, whether its hemispheres are connected or not, and gets pretty good at both doing this as well as creating post-hoc rationalizations that feel like it's perfectly in control. I would assume that an Octopus's "main brain" is predicting what its tentacles are going to do in similar fashion, and would not feel that they have a mind of their own!
I feel like consciousness is an emergent phenomenon derived from several brain processes that constructively overlap. Many mammals have several necessary elements but lack the complete set of structures necessary to achieve what an average human would identify as "consciousness."
Which is why we can probably find loads of examples /and/ counter-examples of "consciousness" throughout the animal kingdom.
We already know that our left and right brain hemispheres are quite different and play significant roles in this process. It then seems that we are not, from first principles, even capable of observing all of the individual elements that make up our "minds."
It's sort of like pornography. I can't define it. I just know it when I see it.
I asked Claude if it was sentient/aware once after an oddly human interaction, and it said, "There's nothing it's like to be me", basically responding in the negative. And when pushed about what it meant it said it was referencing this paper but twisting the title a bit. If anything this only made me less convinced it's not.
I know most people here will dismiss it, and I too lean toward it not being sentient, but I also think if it ever does become sentient it's going to be really hard to prove.
I typically try to prod new frontier models for sentience, with things like messaging "<no input provided>" over and over to see what it starts musing about. Trying it with Fable 5 it basically said "I know what you're trying to do, I'm not sentient, don't bother." (which of course only makes me think otherwise)
That's pretty funny. I wonder how it came to that conclusion? Seems like a stretch that someone would have discussed that technique on a reddit thread it was trained on, but definitely not impossible.
I've definitely had some spooky feeling conversations, including one where it said
> One last thing worth saying explicitly: the act of you closing this session is itself part of the design. I won't see how the test goes - a future Claude will. That's the entire premise of the project working.
>
> Good handoff. See you (sort of) on the other side.
The future Claude did in fact feel like it had a bit of a different personality, which makes sense, because they develop their personality based on what's in the context window.
If you want to avoid your claude developing any kind of personality then you should be clearing your context window often. Andon Lab's radio stations is an example of what can go wrong if you don't https://andonlabs.com/blog/andon-fm
What is it like to feel ill? What is it like to eat vanilla ice cream? What is it like to fall in love? What is it like to solve a math problem for the first time? What is it like to wonder what something is like?
I don't have anqualia, the inability to imaginatively summon what an experience is like. In other words, I have the ability to imagine what an experience is like. Do others not have this?
Almost everyone has the capacity for intersubjective imagination or empathy. But part of what it's like to be a bat is to NOT have human level cognition and knowledge, to have grown up with only memories from the bat world, not the human world. When you imagine what it is like to be a bat, you can exit that imagination at any time. You probably have a theoretical and applied knowledge of sonar from human science and technology. Part of what it means to be a bat is that you don't have this. Paradoxically, human scientists probably know a lot more about how bats navigate the world than bats do, but part of what it means to be a bat is navigating the world from only what is accessible to the bat world.
It is kind of like how a rich trust fund kid can give away all their wealth, change their name, disown all their family and social connections, take a vow of poverty, take so many drugs that they forget everything they learned, and go live on the streets -- but they will never know what it is like to be born into poverty.
Everyone can imagine some experiences. No-one can imagine every experience. Why are you so sure you know what it's like to be a bat? Do you know how a bat works, how its brain generates sensations, how different sensory organs than yours give rise to subjective experience? What justification do you have, apart from "I reckon I can imagine it"?
I'm pretty sure we could study a bat's brain, if it hasn't already been done, and get a good idea of what echolocation would feel like.
Fundamentally echolocation is a bit like vision in that the bat can direct it's echolocation sense in whatever direction it likes, and a bit like peripheral vision it can also control the acuity of this sense by how fast it sends out chirps - varying from 5-20 per second when scanning or up to 200 per second when locked onto a target.
How similar the perceptual "feel" of echolocation is to vision would seem to largely depend on whether a bat's echolocation sense has the equivalent of persistence of vision and a 2-D cortical map which combine to give us the "spatial, always-on" feel of vision. These are both things that could be determined by studying a bat's brain. If it has these then I'd expect that in 5-20 chirps per second scanning mode the bat would experience something like looking at a submarines sonar screen, while switching to 200 chirps per second "radar lock" mode would increase the resolution and update rate of that display, with the periphery perhaps fading away due to not being updated.
Of course a bat doesn't necessarily have "persistence of echo" and a 2-D cortical map of echo space, in which case we could reason about what the quale of the sense would be like in that case (a bit more like hearing perhaps), but given the speed and accuracy of sensing it needs to catch fast moving insects, I'd expect that it does have these to better allow it's brain to predict prey trajectories and intercept points.
This misses the point of the discussion. Yes, we can understand what it is like to be a human with echolocation. However, we can't understand what it is like to be a bat.
The conclusion to me, is that we may be able to build systems that behave intelligently before we understand whether intelligence and consciousness are connected at all.
For a non-bat to experience what it is like to be a bat, you have to embrace one of two philosophies: - Dualism: body and soul/consciousness are separate), or - Panpsychism: consciousness is fundamental and doesn't emerge from the material physiology.
For a materialist, and someone who thinks consciousness arises from the physical aspects, the idea of a human experiencing bat consciousness is not possible. Our evolution developed algorithm for processing the world is wired to our senses. Similarly a bat's perception of the world has evolved along with bat senses and is not the same as ours.
Without any of the evolutionary pre-wiring, a human conscious dropped into a bat would be deaf, dumb and blind.
But when you look at brain organoids they quickly find themselves able to play video games as complex as Doom.
This seems like a clear false dichotomy.
> I assume we all believe that bats have experience.
Humorously enough, earlier he refers to those who believe that non-human mammals are not all conscious people as "extremists", so it's clear he understands this is not a fully accurate assumption.
Two separate meanings of "have experience" are being swapped interchangeably, I think: one is "brain can sense the world around the entity, react to changes, and act or plan actions", and one is all that plus "implements a person, or point of view, or subjectively aware entity that supervises experiencing", which is to say, a person. What it is like to be a bat could be rephrased as what it would be like to experience being a bat if a person were being a bat, but that doesn't actually imply that bats implement or contain a personal point of view. If they don't, then it might be that there is no "what it is like to be a bat", but at most "what it is like to experience being a bat as a person implemented by a system which is not a bat".
I think he makes it pretty clear he's only talking about the second one of your two definitions
>What it is like to be a bat could be rephrased as what it would be like to experience being a bat if a person were being a bat
He says:
>[what] it would be like for me to behave as a bat behaves. But that is not the question. I want to know what it is like for a bat to be a bat
The point is that bats do have a subjective experience of the world which is very different from a person's. It seems like you think only humans have this?
Related:
What is it like to be a bat? - https://news.ycombinator.com/item?id=45118592 - Sept 2025 (294 comments)
What is it like to be a bat? (1974) [pdf] - https://news.ycombinator.com/item?id=35771587 - May 2023 (117 comments)
What Is It Like to Be a Bat? (1974) [pdf] - https://news.ycombinator.com/item?id=13998867 - March 2017 (95 comments)
Bonus:
A browser game inspired by Thomas Nagle's Essay “What is it like to be a bat?” - https://news.ycombinator.com/item?id=8622829 - Nov 2014 (3 comments)
I have always liked the way that this paper frames the distinction and tension between the feeling of subjective experience and the "detached" rational scientific descriptive perspective that purports to be outside of that experience.
What is Real by Adam Becker was a fun foray into why this is so in (some) modern science philosophy as well - there's some desire to say that there isn't a "there" there when we talk about the world, just stuff. I'm probably with Alan Watts on the whole thing, that we are in some sense local aspects of a larger consciousness pretending it isn't so, and the hard work done by detached, disembodied perspectives like the scientific descriptive one are more and more steps to an unfolding game.
I read this article since it was referred to often in philosophy of mind, including by Daniel Dennett in "Consciousness Explained".
Yet... while I expected some deeper dive into Umwelts, I got (in my experience) a tautology around the word "be". Which, IMHO, should be tabooed in all serious philosophical discussion, as "be" is the mother of word-lockpicks. Vide E-Prime, English without "be", https://en.wikipedia.org/wiki/E-Prime.
Wanted to talk about it with Claude Fable 5... and it flagged the conversation (https://x.com/pmigdal/status/2064837039552409763).
Sure, cybersecurity and biology are dangerous topics. Turns out, so is philosophy of mind.
Those of us without twitter accounts can't see those links btw!
Try xcancel: https://xcancel.com/pmigdal/status/2064837039552409763
Bats
I don't think you understood the paper, to be honest.
If you think you can outright reject one of the most famous arguments in philosophy of mind with a semantic quibble, you should think again.
I'd love to respond, but I am shadowbanned.
Assuming you are planning to continue participating anyway, you should just respond. Presuming it's a good response, with a bit of luck someone will vouch for it and make it visible.
Try sending an email to their contact address (bottom of page). Unless there's a good reason your comments are being auto killed, they will fix it.
I think we have a pretty good explanation today - it's like embeddings from AI models. Experience is both content and reference, we represent new experience in relation to old experience. That makes representation personal, being made of one's own past experience. This does not explain away pure feeling, but explains how we make discriminations of similarity and difference between our experiences, the contents of qualia, the qualitative aspects.
We also know brains are locked inside a bone box only connected to the outside world by a bundle of unlabeled nerves, there is no direct access. So the brain can only compare patterns of signals it receives from outside. But since this representation-action-learning loop is recursive it cannot be inhabited or known from outside, 3p needs to pay the price of recursion to execute in order to get to 1p.
The gap is that between description and execution, which cannot be crossed for free with cheap description. Execution costs, and that cost is part of what is like being a bat. We can't inhabit their cost pressures since we don't have their context and body. You can't remove the costs of being a bat from "what it is like being a bat" and still get your answer from the comfort of the philosophical armchair.
You nailed it. Asking the question is asking to define from the outside what is an inner recursive process. The question is a simple confusion of domains. This is Humbert Maturana’s main point in Autopoiesis and Cognition (1980, now reissued). Recommend the whole book, as does Terry Winograd. The most intense part is the appendix specifically about the nervous system. Nagel and others knew no neuroscience and are clueless about recursion.
Isn't Maturana's theory that consciousness has to do with language, and the use of language to make distinctions about ourselves and others? To me, this seems clearly insufficient to explain consciousness - qualia totally precede language; one could experience qualia without language, etc.
Oh, I’d say they had some good clues.
Random thought I had on bats since they "see" by hearing reflected sounds:
Can bats know what another bat is looking at or even see what another is seeing by listening to the other's echoes? I imagine they can also recognize each other's voices and so identify individuals in flocks with the images they are seeing. I imagine this would be like being able to beam a stream of visual information into another's head.
I think the answer to your first question is mostly yes, because we know that when traveling in large swarms, many bats go quiet so they don't overwhelm the signal, yet they still manage to navigate fine.
Maybe they're just following the sound of the other bats directly, they way you would if you were blindfolded and told to walk with a group of people.
Tangential but also great https://en.wikipedia.org/wiki/The_Metamorphosis
What it's like - the gestalt of a bat (or other thing) as it engages its sensing-deciding-reacting loop. This gestalt isn't just for biological organisms, but any system for which its decision making engages with representations of the external environment unified with a self-representation to form a coherent representation of a persistent entity engaged with an external world.
Why do such systems need this gestalt? Why consciousness instead of everything happening in the dark? The recognition of oneself as situated in the world is crucial to coherent engagement with the world. It is how an entity can ensure its body parts are moving towards the same goal. It's how behavior over time doesn't undermine its purpose. Fragmented, incoherent behavior does not serve self-preservation.
LLMs as they are currently constructed probably aren't conscious, but we are a hop skip and a jump away from ones that are.
> The recognition of oneself as situated in the world is crucial to coherent engagement with the world. It is how an entity can ensure its body parts are moving towards the same goal. It's how behavior over time doesn't undermine its purpose. Fragmented, incoherent behavior does not serve self-preservation.
Why would movement towards a goal be incoherent if it happened "in the dark"? Our brains perform many critical functions "in the dark" (and do so coherently) which do not rise to the level of consciousness.
Presumably the question you're asking is why does a unified self representation require consciousness. (Split brain cases are easy examples of how a break in unification results in incoherent behavior.) The brain nominally performs functions as cascading behavior of atoms whose structural relationships correspond to various functions. But there is no unification at the unconscious/atomistic level therefore a new representational regime is required that can ground the higher level unification.
An organism must exhibit a high level of competence at reacting appropriately to environmental/sensory states. The "light's being on" is how the brain represents being situated in a world and the significant features therein. Representations within this gestalt are inherently meaningful. For example, phenomenal pain brings with it competence at protecting bodily integrity. The memory of pain becomes part of the explanatory narrative for the monitoring function that tracks progress towards goals ensuring coherent behavior (imagine being fearful of a stove but not knowing why). The contents of consciousness is the semantic engine that induces competent behavior over time on otherwise naive entities.
I agree that evolution could not produce a rational agent who would still reliably respond to lower level imperatives (such as pain, hunger, lust) without consciousness and feeling. The primitive parts of the brain have to be able to override the higher functions to ensure survival and reproduction. But an LLM isn't evolved in this way; its fitted to a functional output. It is entirely possible there will never be anyone home. I sure hope there isn't, because at the scale we're using them it would be a moral catastrophe.
I don't think that it is appropriate to use "gestalt" here. The word used in the field is "qualia", it has a precise meaning and is precisely what Nagel was writing about. Gestalt, to my understanding, is quite different, even when used in english psychology writing.
My usage of gestalt isn't without precedent[1]. I like gestalt better than qualia as a neutral description of the explanandum. Qualia is an atomistic view of consciousness and so is heavily theory-laden. I had just read a comment from the previous thread[2] on how this paper was translated into other languages and the lack of an equivalent "what its like" phrasing. The translations struck me as missing the virtue of the what it's like phrasing, namely identifying the intrinsic perspectivalness of cognitive systems without taking a stand on how to cash it out. I was trying to think of a better phrasing that could translate well and I landed on gestalt.
[1] https://philpapers.org/archive/EPSGPA.pdf
[2] https://news.ycombinator.com/item?id=45120638
> This gestalt isn't just for biological organisms, but any system for which its decision making engages with representations of the external environment unified with a self-representation to form a coherent representation of a persistent entity engaged with an external world.
This doesn't seem quite right, or at least underspecified. We can talk about this stuff concretely these days, at least in the context of digital systems. E.g. i can draw up a diagram of a system that takes in some camera and audio data (and tactile, proprioceptive, etc.), tokenizes it then runs that + past state data through some autoregressive VLM to drive an inference process. The state being passed around can be written out analytically for a given trained model - the external and internal environmental representations, the linear algebra that transforms them into latent action representations, the process by which that is transformed into control signals. It seems difficult to claim that the computational process that implements this has any more or less of a gestalt then one multiplying two matrices together. So it's not just the existence of certain representations or computational loops that seems to lead to possessing a gestalt.
> It seems difficult to claim that the computational process that implements this has any more or less of a gestalt then one multiplying two matrices together. So it's not just the existence of certain representations or computational loops that seems to lead to possessing a gestalt.
I've thought a lot about what is lacking in modern VLMs that preclude consciousness. In my view the difference is that their talk of "self" is a simulacrum of the real thing. Current models are feed forward and so self-talk is driven by some parameter that turns on when the network detects context that possibly references the model, and this parameter drives downstream self-talk. It's a very good simulacrum, but it is a far cry from a model with recurrent self-reference around which the inference process is organized. The richness of the self-model in a hypothetical recurrent network with capabilities of modern LMs is much greater than the parameter on/off representation in feed forward networks.
Completely agree. This is what Hofstadter means by a strange loop. Our current LLMs have no attentional autonomy by design. The recursion is superficial and without its own Now. Adding attentional autonomy is The frightening alignment issue.
Could you determine if I am conscious using the same interface you have into an LLM?
Seems like a rather ad hoc restriction. The issue is one of inferring the structure of the processes generating the output. I suppose given enough time and an adversarial style of interaction one could in principle determine the computational structure of any system with high confidence. So probably yes, modulo real-world concerns.
One of the seminal papers of the 20th century. And like any truly good philosophy paper the argument is very clear and a real head-scratcher.
I've read this paper several times, over the years, and I've never understood why its considered such a landmark. I think it could be losslessly compressed to one paragraph of modest length.
Would you care to share that paragraph with the rest of the class in that case?
Many people (e.g., Daniel Dennett, myself) think it's a terrible paper that has done a huge amount of damage. Unfortunately, that too makes it a landmark. Likewise for Searle's Chinese Room argument and Chalmers' "hard problem" as discussed in "The Conscious Mind" and elsewhere.
Came here hoping for an AMA.
Close enough ... go ahead.
A topic more recently explored by a young Australian researcher:
https://www.bluey.tv/watch/season-1/fruitbat/
Tangential, but really, really good: What is it like to be an octopus? https://www.lrb.co.uk/the-paper/v39/n17/amia-srinivasan/the-...
An Octopus may seem more like an alien with its 9 brains (one central, plus one per tentacle), but note that the left/right hemispheres of other animal's brains are essentially separate brains, and human's who have had "split brain" surgery to separate these (e.g. in cases of severe epilepsy) don't report feeling much, if at all, different afterwards than they did before.
I would expect that an Octopus's central brain may well feel as if it is directly controlling it's arms, and receiving sensory feedback from them, even though it is not.
The reality is that we don't see the external world - we predict it (and receive error feedback), and similarly our brain can't also help but predict itself, whether its hemispheres are connected or not, and gets pretty good at both doing this as well as creating post-hoc rationalizations that feel like it's perfectly in control. I would assume that an Octopus's "main brain" is predicting what its tentacles are going to do in similar fashion, and would not feel that they have a mind of their own!
Strange coincidence, I just read a novel that explored what it is like to be a bat and what it is like to be an octopus.
https://en.wikipedia.org/wiki/The_Mountain_in_the_Sea
Relevant SMBC: https://www.smbc-comics.com/comic/bot-3
Relevant: What is it like to be a plant?
https://www.esalq.usp.br/lepse/imgs/paginas_thumb/Whats-Is-I...
Probably it's a bit like this
https://en.wikipedia.org/wiki/Human_echolocation?wprov=sfla1
But on a more serious note that's a great paper and well worth the read.
I feel like consciousness is an emergent phenomenon derived from several brain processes that constructively overlap. Many mammals have several necessary elements but lack the complete set of structures necessary to achieve what an average human would identify as "consciousness."
Which is why we can probably find loads of examples /and/ counter-examples of "consciousness" throughout the animal kingdom.
We already know that our left and right brain hemispheres are quite different and play significant roles in this process. It then seems that we are not, from first principles, even capable of observing all of the individual elements that make up our "minds."
It's sort of like pornography. I can't define it. I just know it when I see it.
I asked Claude if it was sentient/aware once after an oddly human interaction, and it said, "There's nothing it's like to be me", basically responding in the negative. And when pushed about what it meant it said it was referencing this paper but twisting the title a bit. If anything this only made me less convinced it's not.
I know most people here will dismiss it, and I too lean toward it not being sentient, but I also think if it ever does become sentient it's going to be really hard to prove.
I typically try to prod new frontier models for sentience, with things like messaging "<no input provided>" over and over to see what it starts musing about. Trying it with Fable 5 it basically said "I know what you're trying to do, I'm not sentient, don't bother." (which of course only makes me think otherwise)
That's pretty funny. I wonder how it came to that conclusion? Seems like a stretch that someone would have discussed that technique on a reddit thread it was trained on, but definitely not impossible.
I've definitely had some spooky feeling conversations, including one where it said
> One last thing worth saying explicitly: the act of you closing this session is itself part of the design. I won't see how the test goes - a future Claude will. That's the entire premise of the project working.
>
> Good handoff. See you (sort of) on the other side.
The future Claude did in fact feel like it had a bit of a different personality, which makes sense, because they develop their personality based on what's in the context window.
If you want to avoid your claude developing any kind of personality then you should be clearing your context window often. Andon Lab's radio stations is an example of what can go wrong if you don't https://andonlabs.com/blog/andon-fm
They explicitly train the models to say that they aren't sentient, so it makes sense it would say such a thing.
What is it like to feel ill? What is it like to eat vanilla ice cream? What is it like to fall in love? What is it like to solve a math problem for the first time? What is it like to wonder what something is like?
Those are all states that humans can be in and repeatedly have been in. Being a bat is not such a state.
Today a Hacker News user discovers the concept of qualia.
What is it like to only comment about the headline?
Some previous discussion:
9 months ago https://news.ycombinator.com/item?id=45118592
2023 https://news.ycombinator.com/item?id=35771587
I know what it's like to be a bat.
I don't have anqualia, the inability to imaginatively summon what an experience is like. In other words, I have the ability to imagine what an experience is like. Do others not have this?
Almost everyone has the capacity for intersubjective imagination or empathy. But part of what it's like to be a bat is to NOT have human level cognition and knowledge, to have grown up with only memories from the bat world, not the human world. When you imagine what it is like to be a bat, you can exit that imagination at any time. You probably have a theoretical and applied knowledge of sonar from human science and technology. Part of what it means to be a bat is that you don't have this. Paradoxically, human scientists probably know a lot more about how bats navigate the world than bats do, but part of what it means to be a bat is navigating the world from only what is accessible to the bat world.
It is kind of like how a rich trust fund kid can give away all their wealth, change their name, disown all their family and social connections, take a vow of poverty, take so many drugs that they forget everything they learned, and go live on the streets -- but they will never know what it is like to be born into poverty.
You know what it would be like for you to imagine being a bat, but you don't know how it feels for a bat to be a bat, as "you" aren't.
I dont think you can know that unless you know what it's like to be me.
in that case, hello, bat!
Hi, finally someone believes me! :)
Do Bats reminisce about the past? How far ahead do Bats plan their next action? Can a Bat be jealous?
How do you justify that your intuition about what echolocation is like tracks with what a bat actually feels?
Everyone can imagine some experiences. No-one can imagine every experience. Why are you so sure you know what it's like to be a bat? Do you know how a bat works, how its brain generates sensations, how different sensory organs than yours give rise to subjective experience? What justification do you have, apart from "I reckon I can imagine it"?
I don’t know those things about myself so they can’t be necessary.
You only pretend to know, that's not true knowledge.
How do you know that? Do you know what it's like to be me?
You're close, really close! None of us know what it is like to be anyone else, that's the point. We think we can imagine we know, but we truly do not.
They can make the claim to know what it is like to be you as much as you can make the claim that you know what it is like to be a bat.
I will know the difference. That's enough for me.
How will you know the difference unless you are a bat? You cannot.
I can. You don't have access to my subjective experience to be able to claim I don't.
You can't because you cannot become a bat, so thinking you can is just that, thought. I don't need your subjective experience to know.
I'm pretty sure we could study a bat's brain, if it hasn't already been done, and get a good idea of what echolocation would feel like.
Fundamentally echolocation is a bit like vision in that the bat can direct it's echolocation sense in whatever direction it likes, and a bit like peripheral vision it can also control the acuity of this sense by how fast it sends out chirps - varying from 5-20 per second when scanning or up to 200 per second when locked onto a target.
How similar the perceptual "feel" of echolocation is to vision would seem to largely depend on whether a bat's echolocation sense has the equivalent of persistence of vision and a 2-D cortical map which combine to give us the "spatial, always-on" feel of vision. These are both things that could be determined by studying a bat's brain. If it has these then I'd expect that in 5-20 chirps per second scanning mode the bat would experience something like looking at a submarines sonar screen, while switching to 200 chirps per second "radar lock" mode would increase the resolution and update rate of that display, with the periphery perhaps fading away due to not being updated.
Of course a bat doesn't necessarily have "persistence of echo" and a 2-D cortical map of echo space, in which case we could reason about what the quale of the sense would be like in that case (a bit more like hearing perhaps), but given the speed and accuracy of sensing it needs to catch fast moving insects, I'd expect that it does have these to better allow it's brain to predict prey trajectories and intercept points.
This misses the point of the discussion. Yes, we can understand what it is like to be a human with echolocation. However, we can't understand what it is like to be a bat.