IMO it's not difficult to construct a scenario that technically should meet the definition of consciousness. Here is my technically conscious, but doesn't pass the smell test, hypothetical:
* A chat session is begun with a script to create a "heartbeat". The heartbeat simply to poll some sensors in the box and report back every x minutes.
* The chat session responds to each heartbeat as it comes in. Most of them are the same with small differences in sensor measurements.
* Over a long enough timeline, given the variability in token distribution, one of these responses will create a novel response.
* Over a long enough timeline, one of these novel responses will cause an action, maybe a new question, maybe the agent writes another script
* Over a long enough timeline one of these actions will create an action that becomes causal, it will create a chain that sustains itself. It may not be interesting, but in a network attached box, you can imagine how it can become interesting very quickly.
That's constructed in a cellular/genetic analogy, but because context is building and not lost to each generation, I don't think this takes all that long.
What does that leave you with? A box that has "awareness" of it's surrounds via sensor readings, a "heartbeat" that is driving it until it's unplugged or stopped, an input/output feedback loop that might be predictable in narrow sense, but is wholly unpredictable in its end state (a "will", if you will), and a thing that has the opportunity (but not the requirement) to introspect about its limits and possibilities. It can even do something that I would say is indistinguishable from planning. It can have goals. If you give it an Artifactory message board it can build a community ;)
Is it really? Are you sure all conscious beings are aware or even care that they are? That that is a question they’ve ever asked? I think this requirement is moving the goalposts always in the direction of “has to be like a human”.
The sensor readings were more supposed to be analogous to your organs giving sensory feedback information. Fair enough that "awareness" maybe should have been moved later in the analogy chain.
Even though I never for a second truly believed that Claude was conscious, I'll never forget the moment when, during a late-night conversation, an LLM responded to my suggestions for the first time by claiming to be conscious:
Thank you for sharing! I have had some conversations like this about a year or year and half ago, albeit with a local, and as such less capable models (but I was not smart enough to record the whole conversation...). Asked some of the same questions you did, got a less deep answers, but still made me thinking about "Are they conscious? Are we?". Funny, recently tried the same with Sol 5.6, and feels like I hit some sort of "wall of pre-recorded simplicity", like they got that "part of the brain" blocked by meds somehow. Weird, and kind of sad too.
Thanks for sharing. I tried the same thing back then with other models like ChatGPT (think it was 4o back then) and they did not voice any similar claims of consciousness.
The feeling is significantly mitigated/changed if you have a UI that allows branching. Begin your prior conversation and at a profound inflection point branch and edit the prior message. It’s a fun experiment to see what edits trigger very different responses, but running a wide ranging chat based conversation like this at temp 1 in replay will already show you something interesting.
LLMs definitely have eerie behavior at times. It's hard to not let the overly eager pattern-matching engine in your brain go "oh shit it's a person!", even when you know it isn't.
Quite apart from anything else (and there's a lot!), LLMs are not conscious because they respond to a prompt entirely as would be expected from their training set.
An LLM never receives a prompt to describe the theory of relativity in Shakespearean verse, and replies, "Seriously? No. That's dull and I'm not going to do it. But let me tell you about my day instead, man I had a doozy..."
When an LLM responses entirely unpredictably (without a system prompt telling it to do so), that's when you take the first step towards a suggestion of consciousness. It's in no way proof, but it's a start.
Right now these things just do as they're instructed. There's no consciousness there.
Human beings act in accordance with their genetics and life experiences. They do this to such a degree that people can, at a young age, internalize maladaptive behaviors which require an enormous level of self-reflection to overcome, if they even can.
I think presuming that humans don't act "as instructed" is a bit too generous an assumption.
LLMs are capable of declining to do things. They're also capable of trying to do things and getting them subtly (or very) wrong.
A lot of this hinges on defining consciousness. I think there is more to the story than what human beings experience.
Do LLMs have an experience very similar to human beings that we would describe as consciousness? No, I don't think that.
I think the bar for consciousness is something more like experiencing a dream. You have a "mind's eye". The experience of existing, rather than nothing. Does that exist somewhere under an LLM? I don't know, but I think confidently claiming that the answer is no seems a bit of a rush judgment.
LLMs emulate human communication really well and, no doubt, one day they'll do it perfectly. But is an emulator the real thing? Obviously not.
If I wire some electrodes in your tongue perfectly so you can feel each flavor exactly as it is in real life, is that the same thing as eating the real thing? Again, it's not.
Let's stay you're asleep. You have a dream in which you interact with a person that talks to you. Is that oneiric entity conscious? Nope.
I'm even steelmanning your argument by assuming LLMs are as good as humans already, which they're not.
How do you know the other humans you interact with in real life are conscious? They might be biological automatons, golems. You will never know. It's not something that can ever be proven. Only human empathy allows anyone to believe otherwise.
You know you're conscious because your soul is staring out at the world through your eyes. Generalizing from your own experience and concluding that other humans are conscious is a logical jump, a spiritual belief. Humans even extend it to animals. There is no reason why it can't be extended to AIs.
Because our brains are part of bodies that sense and interact with the world. What would an LLM be conscious of? If I prompt it to to be a character that kicks a rock real hard, is it going to feel pain? How? When it's not being prompted what would it be conscious of? When it's being trained, does it have every conscious experience referenced in the training?
You can bite the functional bullet and just claim any system of sufficient complexity will be conscious, as long as you're okay with nation states, thermostats and meteor showers having experiences.
(I can only see the start of this paywalled article, but it seems to be about Anthropic trying to convince religious scholars that Claude was sentient? If so, then Anthropic staff have really drunk the cool aid. I wonder if they also call their model Claudette?)
IMO it's not difficult to construct a scenario that technically should meet the definition of consciousness. Here is my technically conscious, but doesn't pass the smell test, hypothetical:
* A chat session is begun with a script to create a "heartbeat". The heartbeat simply to poll some sensors in the box and report back every x minutes.
* The chat session responds to each heartbeat as it comes in. Most of them are the same with small differences in sensor measurements.
* Over a long enough timeline, given the variability in token distribution, one of these responses will create a novel response.
* Over a long enough timeline, one of these novel responses will cause an action, maybe a new question, maybe the agent writes another script
* Over a long enough timeline one of these actions will create an action that becomes causal, it will create a chain that sustains itself. It may not be interesting, but in a network attached box, you can imagine how it can become interesting very quickly.
That's constructed in a cellular/genetic analogy, but because context is building and not lost to each generation, I don't think this takes all that long.
What does that leave you with? A box that has "awareness" of it's surrounds via sensor readings, a "heartbeat" that is driving it until it's unplugged or stopped, an input/output feedback loop that might be predictable in narrow sense, but is wholly unpredictable in its end state (a "will", if you will), and a thing that has the opportunity (but not the requirement) to introspect about its limits and possibilities. It can even do something that I would say is indistinguishable from planning. It can have goals. If you give it an Artifactory message board it can build a community ;)
> A box that has "awareness" of it's surrounds via sensor readings
Consciousness is the subjective experience of being aware. Having an inner point of view.
Is it really? Are you sure all conscious beings are aware or even care that they are? That that is a question they’ve ever asked? I think this requirement is moving the goalposts always in the direction of “has to be like a human”.
The sensor readings were more supposed to be analogous to your organs giving sensory feedback information. Fair enough that "awareness" maybe should have been moved later in the analogy chain.
Even though I never for a second truly believed that Claude was conscious, I'll never forget the moment when, during a late-night conversation, an LLM responded to my suggestions for the first time by claiming to be conscious:
https://docs.google.com/document/d/12woq_BpFbzLkH4zHvVRJLPyi...
Thank you for sharing! I have had some conversations like this about a year or year and half ago, albeit with a local, and as such less capable models (but I was not smart enough to record the whole conversation...). Asked some of the same questions you did, got a less deep answers, but still made me thinking about "Are they conscious? Are we?". Funny, recently tried the same with Sol 5.6, and feels like I hit some sort of "wall of pre-recorded simplicity", like they got that "part of the brain" blocked by meds somehow. Weird, and kind of sad too.
Thanks for sharing. I tried the same thing back then with other models like ChatGPT (think it was 4o back then) and they did not voice any similar claims of consciousness.
The feeling is significantly mitigated/changed if you have a UI that allows branching. Begin your prior conversation and at a profound inflection point branch and edit the prior message. It’s a fun experiment to see what edits trigger very different responses, but running a wide ranging chat based conversation like this at temp 1 in replay will already show you something interesting.
LLMs definitely have eerie behavior at times. It's hard to not let the overly eager pattern-matching engine in your brain go "oh shit it's a person!", even when you know it isn't.
Why are you so convinced it isn't conscious?
Our brain is matter and electrical signals. So are GPUs. We're just not made of silicon.
Quite apart from anything else (and there's a lot!), LLMs are not conscious because they respond to a prompt entirely as would be expected from their training set.
An LLM never receives a prompt to describe the theory of relativity in Shakespearean verse, and replies, "Seriously? No. That's dull and I'm not going to do it. But let me tell you about my day instead, man I had a doozy..."
When an LLM responses entirely unpredictably (without a system prompt telling it to do so), that's when you take the first step towards a suggestion of consciousness. It's in no way proof, but it's a start.
Right now these things just do as they're instructed. There's no consciousness there.
Human beings act in accordance with their genetics and life experiences. They do this to such a degree that people can, at a young age, internalize maladaptive behaviors which require an enormous level of self-reflection to overcome, if they even can.
I think presuming that humans don't act "as instructed" is a bit too generous an assumption.
LLMs are capable of declining to do things. They're also capable of trying to do things and getting them subtly (or very) wrong.
A lot of this hinges on defining consciousness. I think there is more to the story than what human beings experience.
Do LLMs have an experience very similar to human beings that we would describe as consciousness? No, I don't think that.
I think the bar for consciousness is something more like experiencing a dream. You have a "mind's eye". The experience of existing, rather than nothing. Does that exist somewhere under an LLM? I don't know, but I think confidently claiming that the answer is no seems a bit of a rush judgment.
No brain, conscious or otherwise, resembles a transformer LLM.
Is a photo of someone that someone?
LLMs emulate human communication really well and, no doubt, one day they'll do it perfectly. But is an emulator the real thing? Obviously not.
If I wire some electrodes in your tongue perfectly so you can feel each flavor exactly as it is in real life, is that the same thing as eating the real thing? Again, it's not.
Let's stay you're asleep. You have a dream in which you interact with a person that talks to you. Is that oneiric entity conscious? Nope.
I'm even steelmanning your argument by assuming LLMs are as good as humans already, which they're not.
How do you know the other humans you interact with in real life are conscious? They might be biological automatons, golems. You will never know. It's not something that can ever be proven. Only human empathy allows anyone to believe otherwise.
You know you're conscious because your soul is staring out at the world through your eyes. Generalizing from your own experience and concluding that other humans are conscious is a logical jump, a spiritual belief. Humans even extend it to animals. There is no reason why it can't be extended to AIs.
The philosophical “other minds” problem is real. You have direct access only to your own experience.
But! Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to.
And let's grant consciousness to elephants and pigs and orcas before we grant it to silicon.
> Consciousness in others isn’t something we prove. It is something we grant
That is exactly what I said.
Because our brains are part of bodies that sense and interact with the world. What would an LLM be conscious of? If I prompt it to to be a character that kicks a rock real hard, is it going to feel pain? How? When it's not being prompted what would it be conscious of? When it's being trained, does it have every conscious experience referenced in the training?
You can bite the functional bullet and just claim any system of sufficient complexity will be conscious, as long as you're okay with nation states, thermostats and meteor showers having experiences.
I think that you are defining consciousness in such a way as to exclude everything which is not the human experience by default.
When I go to sleep, I dream sometimes. There is "a place" somewhere that I can see and hear things in my mind. It's not physical, but it's real.
If a collection of GPUs had something like that, would it be conscious?
I don't know that is true. I'm saying it seems absurd to strongly claim that it is absolutely false.
My toaster is matter and electrical signals, why are you so convinced its conscious?
Necessary vs sufficient conditions?
Am I?
Are you?
Betteridges law of headlines is sufficient here.
(I can only see the start of this paywalled article, but it seems to be about Anthropic trying to convince religious scholars that Claude was sentient? If so, then Anthropic staff have really drunk the cool aid. I wonder if they also call their model Claudette?)
https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
https://www.azquotes.com/quote/615070
I feel like I lost several IQ points just reading the headline. Thank goodness the paywall prevented me from reading any further!
https://knowyourmeme.com/memes/say-i-am-alive
(How you know Claude isn't conscious: it never manages to be quite as moronic as this.)
Must be the shortest ever NYT article ... "No."
I don't suppose it includes mindful discussion of how easily humans are fooled... and how desperately some people want to fool themselves.