The problem hasn't been solved, and, by the very nature of what LLMs are, can't be. However, it has gotten a lot better, as models have got much larger and pretraining more sophisticated.
Part of the difficulty--not in solving, but in discussing--is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.
I prefer to say that it's all hallucinations so the time... we just happen to like some portion of them.
_______
> In fact [The Bursar] was incurably insane and hallucinated more or less continuously, but by a remarkable stroke of lateral thinking his fellow wizards had reasoned that, in that case, the whole business could be sorted out if only they could find a formula that caused him to hallucinate that he was completely sane.
> This had worked well. There had been a few false starts. For several hours, at one point, he had hallucinated that he was a bookcase. But now he was permanently hallucinating that he was a bursar, and that almost made up for the small side-effect that also led him to hallucinate that he could fly.
We're being forced to use some AI features in our IDE at work these days, and we see it make stuff up all the time. Very unpleasant to work with. It will tell us implementing something a certain way is impossible, when I've implemented it myself that way in the past. It will fabricate reasons why builds are failing. It will make assumptions about database schemas from thin air.
It's like a science fiction writer or an improv actor doing technobabble. If by chance it knows the actual answer it might use it, but even if not it still has to say _something_ that sounds plausible to a layperson. It'll never say it doesn't actually know, because their character it's acting as _should_ know.
It hasn't and likely never will because it is a structural part of how AI works.
You can see proof of this if you ask it obscure enough questions. That doesn't mean obscure scientific questions, I asked it questions regarding ship fits and modules in EVE Online, which is an extremely well-documented videogame. There are hundreds of online tools to help you for different things, mining yield calculators and more.
Well, ChatGPT just made up almost everything. Very confidently. It couldn't even get the damage types right, I was quite shocked.
Hallucination has only be solved for extremely narrow sets of problems, and only partially. Coding is one of those problems.
Sorry to be flippant but when somebody makes their argument based on their experience with “ChatGPT” I automatically assume their opinion is completely unqualified. ChatGPT is not a model, and your comment gives no indication of when this happened. Whether it happened last week or last year makes a big difference. As does how ChatGPT wraps the LLM and how you interact with it.
From my experience it has been largely solved for one significant use case which is chatting to frontier models about the reality as described by public knowledge. 2 years ago models would rely on their training data, today they go out of their way trying to look it up on the Internet and verify thoroughly. I have not had a problem for a very long time.
When working off of limited, private/unverifiable context, LLMs still hallucinate, but again, much less than 2 years ago, and more within a "getting confused where a human would easily get confused" range, rather than "outrageously making things up" range.
I'd agree with other posters - I'm seeing less hallucination; probably the model-makers are tuning to reduce that, because people make a big deal about it.
I see it as a small deal - it reminds us to check the responses against references.
LLMs are a statistical construction, and everyone accepts without complaint that statistical models have a predicted false-positive and false-negative rate.
I think of "hallucination" as a false positive; false negative is no answer when the model could have made a useful response.
I think there's some relation between our creativity and the LLM capability we dismissively call hallucination.
There are two common ways to spot hallucinations. Talk to it about a topic you know well, or implement what it suggests and get an error or failure.
For the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)
The second still happens, but with much less regularity for me though. It does make mistakes though.
In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.
I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?
This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.
AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given. That's real. Right now. That that is happening anywhere on the planet scares me. That's why I keep asking about AI hallucinating, and it's implications when the stakes are life and death.
The "AIs" being used by these drones have little to do with ChatGPT at the moment. And you should be more worried about "AIs" that do aim at the intended target, anyways.
I don't have a problem with hallucinations, rather the confabulations.
We want AI to make predictions under uncertainty that could be wrong. What we don't need is getting know facts wrong.
One of the best responses I got from ChatGPT was when it said "I don't know", and on questioning it - it responded that it was an unsolved problem and it couldn't objectively take a side iun the argument.
You have to be specific. You only have vague comments about your own thoughts. We had this discussion with Tomahawks in the ninetees, it is not an easy one. This is like answering how long is a string.
IMHO This feels like a troll, OP have several comments about evil Ukrainian AI drones. I feel it is hard to have a naunced discussion if you only diss one part.
Your question is asking for easy technical opinions about a subject where you seem to have no technical knowledge.
We can check it by asking it about the topics we knew. If it says correct many times , then it has less chance of assuming that this model hallucinates. If it gives wrong answer many times we can take it as frequently hallucinating model.
Gemini is the only model that I still see regularly flat-out hallucinate, e.g. the other day it told me we were using a particular technology (Fivetran) on a project when we aren’t and I have literally no idea where it could have gotten that from. No mention of it at all, etc.
Claude will make mistakes but they’re largely “reasonable”. In some ways that’s worse as they’re more believable.
GPT I can’t comment on too much, except to say we run a chatbot using it at my work and when extracting data from the conversation (contact info and such) it will occasionally make up an email address that wasn’t entered into the chat. We have guardrails around it, so it’s not a big deal, but it does happen.
AI will never be fully reliable, and people never will be. But AI is wrong in much less predictable and stable ways, and with much less accountability.
I'm hoping for more than anecdotes. I believe AI is not ready for what it's being used for, but all I get are downvotes and no citations. Maybe all of you are just scared to say it out loud?
No one is scared of saying something anonymously online, and wondering if that’s what’s going on is immature and silly. People earnestly disagree with you, but they don’t owe you an argument
I think its solved, with the right setup and model. I couldn't tell u the last time my agent hallucinated. For me I consider it solved. However some dude feeding massive docs into gpt chat and long conversations, It is not solved in this context.
It matches the reality of my experience using a good frontier like Sol on high to ultra thinking. If it has any way to verify its work whatsoever, it eventually gets to a working and sanely-engineered solution. Blatant hallucinations making it to the final stages have become extremely rare in my use cases, nonexistent if the model has a valid feedback loop. So yes, I will insist someone is likely "holding it wrong" if they still think SOTA AI is spewing out garbage at this stage when I can have it do something like write entire working kernel module fixes for old MacBooks on a whim with no crashes or flaws observed after months of use.
I think this is the issue others are having, when you say:
> when I can have it do something like write entire working kernel module fixes for old MacBooks on a whim
They’re not saying it can’t do that, and that’s not proof it doesn’t hallucinate. In fact, having used 6-8 agents at a time for a year plus while writing AI tooling for an AI startup, I can definitely surely tell you that they’re almost inversely correlated as in models that hallucinate a lot sometimes also put out the best most impressive solutions.
I’m definitely not anti AI and I definitely have found a way to make it work very well and I’m content with the work I get out of it (again maxing out several max 20x subs), but I have had sol definitely hallucinate this week and I’m a bit shocked you’re trying to say otherwise.
Listen I know it’s going to be I’m holding it wrong too, but I’ve been reading white papers and research on LLMs for a long time and was definitely at the cutting edge of context engineering, implementing features in our tooling harness a year before they were in codex or Claude.
maybe I am holding it wrong still but but like at some point if I’m holding it wrong who else will be holding it right? Dozens of people? At some point, the technology has to be approachable enough for everyone to have your point of view automatically.
What do we mean by hallucinate? I'm not counting it making a mistake that it fixes on its own without intervention.
I'm no expert on the inner workings/harnesses/etc beyond a basic understanding of the architecture. Maybe I've just developed a good sense for effective prompts? I could share some recent sessions.
Would you trust it to make life and death decisions? Because it is being used in that context. Drones with AI, armed, and with discretion to choose a target and kill it.
The types of people using it that way are not concerned with the best interests of anyone but themselves, and arguably not even that except within a very immediate time horizon.
What makes you think I'm not concerned about it? I've been positively mortified at the seeming cavalier way in which all caution is thrown to the winds. I've honestly had to fundamentally reassess my understanding of the average character of humanity given the last decade.
Fact is, those with the foresight to see how this can go badly are also seemingly the types of people who don't end up in a position to prevent harms by it. Furthermore, it seems inevitable in a sense, because greed for power seems to necessitate development of automated weapons in ASAP in spite of the risks, and the fact it basically renders traditional warfare pointless.
It appears we'll have to learn the hard lessons, same as our forebearers with. I just hope we can avoid having to regress back to sticks and stones on account of fucking ourselves by overdoing our capability to destroy on account of not being willing0able to peacefully coexist.
No, because that's Terminator territory (ULAWS). Since drones' inception, US military required an officer to approve lethal weapon release until at least 2013 and it may or may not still be required. The official policy is DODD 3000.09, which uses the vague word "appropriate" 22 times, but doesn't allow unsupervised lethal autonomous (ULAWS) yet. Maybe someone knows what the criteria are used around in the world's militaries these days beyond Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy (2023-2024) (which Ukraine signed) (There's a competing REAIM 2023 Call to Action that Ukraine didn't sign with a note.[0])
In other news, New Orleans 911 is using AI to triage localized incident duplication calls from unique emergencies. I'm not saying that it's good or their only practical choice, but it's happening.
The biggest dangers I see are the outsourcing of supervisory control, appeal to authority (when used to summarize content or answer a question), and hallucinated mistakes.
My personal opinion: all output from LLMs is hallucinations. The idea that it’s “right” or “wrong”, when stating factual information is really in the eye of the beholder/user. What year did astronauts land on the moon? — is a question that may or may not have a factual answer depending on your own beliefs. Philosophically, you would need to also define what a fact is, or what “not hallucinating” is, to define what a hallucination is. My understanding is that this property of the LLM architecture is innate, and until we have “world model” LLMs, or some other model that reasons from first principles, instead of the current “guess the next word” model, this isn’t going away. Just don’t rely on the for facts.
Is this distinct from the hardware in a land mine making the decision to explode shrapnel through a human being without any human in the chain except the dead guy.
Worth noting that both cases indirectly involve the humans that designed devices and the humans that made the placement and trigger condition decisions.
Further:
> AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given.
are remote operated, they allow defenders cover while themselves being out and exposed.
However were they altered to autonomously fire, that would be on the basis of pattern matching in the visible and infra red spectrum - shoot at all hot blobs.
That's more of a trigger threshold setting issue than an LLM hallucination issue, and the danger is on par with any weapon system on auto fire, you really shouldn't approach such things until they are put in a safe off state or have exhausted ammunition.
I guess what I'm really asking is whether or not it's moral and ethical to use AI that targets autonomously? My understanding is that it's being used that way, but I can't back that up and I'll take your word for it if that's not the case. But it is certainly a plausible scenario. Everyone invested in AI seems determined to put it in everything because they need consumers to want to pay for it, and right now they haven't figured out a way to earn back from consumers the over a trillion dollars that has been put into the AI bubble. I don't have much faith that the people with billions invested don't want return on their investment, and I don't think they care about the consequences necessary to get it back.
> moral and ethical to use AI that targets autonomously?
First point, vision systems have been used in industry to look for misaligned labels, incorrectly folded papers (in high speed paper presses), wrong items on high speed conveyor belts etc. for thirty odd years now - they have issues that a very distinct from LLM 'AI' hallucinations.
That's nomenclature out the way.
Landmines are indiscriminate, they trigger on any weight or pressure over a threshold.
A vision based Felixer, by contrast, only triggers on cats (well, almost always only) and leaves bilbies and bettongs to walk on by.
That's an improvement over landmines.
The crux of your issue here might be the morality and ethics of establishing human exclusion zones within which all humans are highly likely to die.
These historically are created with rapid patterned artillery fire, butterfly mines, Napalm, indiscriminate criss crossing machine gun fire, etc.
Now there exists an option to use drones to kill all humans and leave the horses and cows alive.
Is it your concern that a bad vision threshold might kill a horse rather than a person? (Likely not)
Would you prefer an area to be napalm'd and agent orange'd back to dust?
Also, why do people keep talking about landmines? I'm talking about software, you are talking about dumb mechanical things from WWII. This isn't a philosophical question about methods of warfare from the past, but the software that controls the dumb hardware.
I, a person, singular, mention landmines as they are a hardware device that are designed to trigger action on a threshold.
Felixers are computer vision based devices that are designed to trigger action on a threshold.
These are actual real world objects that Bishop Berkeley can kick, not vague philosophical questions but actual engineered devices that kill and are in use today.
The problem hasn't been solved, and, by the very nature of what LLMs are, can't be. However, it has gotten a lot better, as models have got much larger and pretraining more sophisticated.
Part of the difficulty--not in solving, but in discussing--is in defining what a hallucination is. On the face of it, it seems straightforward: an obviously counterfactual claim or manifest error of reasoning. However, it's not always that simple. A lot of what people consider to be hallucinations are misattributions, specious diagnoses, strangely lopsided preoccupations, eccentric design choices, needlessly verbose or circuitous output or explanations, a kind of metaphysical conflation of the trivial with the significant, etc.
I prefer to say that it's all hallucinations so the time... we just happen to like some portion of them.
_______
> In fact [The Bursar] was incurably insane and hallucinated more or less continuously, but by a remarkable stroke of lateral thinking his fellow wizards had reasoned that, in that case, the whole business could be sorted out if only they could find a formula that caused him to hallucinate that he was completely sane.
> This had worked well. There had been a few false starts. For several hours, at one point, he had hallucinated that he was a bookcase. But now he was permanently hallucinating that he was a bursar, and that almost made up for the small side-effect that also led him to hallucinate that he could fly.
—The Truth (2000) by Terry Pratchett
We're being forced to use some AI features in our IDE at work these days, and we see it make stuff up all the time. Very unpleasant to work with. It will tell us implementing something a certain way is impossible, when I've implemented it myself that way in the past. It will fabricate reasons why builds are failing. It will make assumptions about database schemas from thin air.
It's like a science fiction writer or an improv actor doing technobabble. If by chance it knows the actual answer it might use it, but even if not it still has to say _something_ that sounds plausible to a layperson. It'll never say it doesn't actually know, because their character it's acting as _should_ know.
It hasn't and likely never will because it is a structural part of how AI works.
You can see proof of this if you ask it obscure enough questions. That doesn't mean obscure scientific questions, I asked it questions regarding ship fits and modules in EVE Online, which is an extremely well-documented videogame. There are hundreds of online tools to help you for different things, mining yield calculators and more.
Well, ChatGPT just made up almost everything. Very confidently. It couldn't even get the damage types right, I was quite shocked.
Hallucination has only be solved for extremely narrow sets of problems, and only partially. Coding is one of those problems.
Sorry to be flippant but when somebody makes their argument based on their experience with “ChatGPT” I automatically assume their opinion is completely unqualified. ChatGPT is not a model, and your comment gives no indication of when this happened. Whether it happened last week or last year makes a big difference. As does how ChatGPT wraps the LLM and how you interact with it.
There has been quite significant progress.
From my experience it has been largely solved for one significant use case which is chatting to frontier models about the reality as described by public knowledge. 2 years ago models would rely on their training data, today they go out of their way trying to look it up on the Internet and verify thoroughly. I have not had a problem for a very long time.
When working off of limited, private/unverifiable context, LLMs still hallucinate, but again, much less than 2 years ago, and more within a "getting confused where a human would easily get confused" range, rather than "outrageously making things up" range.
I'd agree with other posters - I'm seeing less hallucination; probably the model-makers are tuning to reduce that, because people make a big deal about it.
I see it as a small deal - it reminds us to check the responses against references. LLMs are a statistical construction, and everyone accepts without complaint that statistical models have a predicted false-positive and false-negative rate. I think of "hallucination" as a false positive; false negative is no answer when the model could have made a useful response. I think there's some relation between our creativity and the LLM capability we dismissively call hallucination.
There are two common ways to spot hallucinations. Talk to it about a topic you know well, or implement what it suggests and get an error or failure.
For the first, we're mostly past the point of just "testing". So I don't see that too much anymore. Mostly I still see that though in bug reports generated by AI by someone else. There is usually some underlying bug being reported, but the AI explanation and "helpful suggestion" is typically inaccurate. Generally, suggested fixes are terrible. (They likely work, but fix a symptom not the cause.)
The second still happens, but with much less regularity for me though. It does make mistakes though.
In areas where I'm not as skilled it's very hard to spot errors. When researching general information I'm mostly accepting it on face value.
I find the bug-report thing really interesting. For lots of simple bugs it's great. For more complex things it seems to be very superficial- if a 0 causes an issue here, add a simple guard for 0. There's no depth of understanding why the value is 0 in the first place, when it should be set. If it can (incorrectly) be 0 here, where else might 0 be impacting the code?
This informs my opinion of vibe coded stuff - where there is no skilled human inspection. I expect that code to be of a poor underlying quality. Especially if it's AI changes to an existing human-coded app.
AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given. That's real. Right now. That that is happening anywhere on the planet scares me. That's why I keep asking about AI hallucinating, and it's implications when the stakes are life and death.
The "AIs" being used by these drones have little to do with ChatGPT at the moment. And you should be more worried about "AIs" that do aim at the intended target, anyways.
I don't have a problem with hallucinations, rather the confabulations.
We want AI to make predictions under uncertainty that could be wrong. What we don't need is getting know facts wrong.
One of the best responses I got from ChatGPT was when it said "I don't know", and on questioning it - it responded that it was an unsolved problem and it couldn't objectively take a side iun the argument.
You have to be specific. You only have vague comments about your own thoughts. We had this discussion with Tomahawks in the ninetees, it is not an easy one. This is like answering how long is a string.
IMHO This feels like a troll, OP have several comments about evil Ukrainian AI drones. I feel it is hard to have a naunced discussion if you only diss one part.
Your question is asking for easy technical opinions about a subject where you seem to have no technical knowledge.
We can check it by asking it about the topics we knew. If it says correct many times , then it has less chance of assuming that this model hallucinates. If it gives wrong answer many times we can take it as frequently hallucinating model.
Gemini is the only model that I still see regularly flat-out hallucinate, e.g. the other day it told me we were using a particular technology (Fivetran) on a project when we aren’t and I have literally no idea where it could have gotten that from. No mention of it at all, etc.
Claude will make mistakes but they’re largely “reasonable”. In some ways that’s worse as they’re more believable.
GPT I can’t comment on too much, except to say we run a chatbot using it at my work and when extracting data from the conversation (contact info and such) it will occasionally make up an email address that wasn’t entered into the chat. We have guardrails around it, so it’s not a big deal, but it does happen.
AI will never be fully reliable, and people never will be. But AI is wrong in much less predictable and stable ways, and with much less accountability.
I'm hoping for more than anecdotes. I believe AI is not ready for what it's being used for, but all I get are downvotes and no citations. Maybe all of you are just scared to say it out loud?
No one is scared of saying something anonymously online, and wondering if that’s what’s going on is immature and silly. People earnestly disagree with you, but they don’t owe you an argument
I'm asking a legitimate question, and only two people have actually tried to make an argument.
Most, like you, have nothing useful to contribute.
please explain how you have discerned that people earnestly disagree with me?
[flagged]
[dead]
[flagged]
I have t see t hat I a few years
I think its solved, with the right setup and model. I couldn't tell u the last time my agent hallucinated. For me I consider it solved. However some dude feeding massive docs into gpt chat and long conversations, It is not solved in this context.
Man there really isn’t a criticism that won’t lead to a booster saying you’re holding it wrong
It matches the reality of my experience using a good frontier like Sol on high to ultra thinking. If it has any way to verify its work whatsoever, it eventually gets to a working and sanely-engineered solution. Blatant hallucinations making it to the final stages have become extremely rare in my use cases, nonexistent if the model has a valid feedback loop. So yes, I will insist someone is likely "holding it wrong" if they still think SOTA AI is spewing out garbage at this stage when I can have it do something like write entire working kernel module fixes for old MacBooks on a whim with no crashes or flaws observed after months of use.
I think this is the issue others are having, when you say:
> when I can have it do something like write entire working kernel module fixes for old MacBooks on a whim
They’re not saying it can’t do that, and that’s not proof it doesn’t hallucinate. In fact, having used 6-8 agents at a time for a year plus while writing AI tooling for an AI startup, I can definitely surely tell you that they’re almost inversely correlated as in models that hallucinate a lot sometimes also put out the best most impressive solutions.
I’m definitely not anti AI and I definitely have found a way to make it work very well and I’m content with the work I get out of it (again maxing out several max 20x subs), but I have had sol definitely hallucinate this week and I’m a bit shocked you’re trying to say otherwise.
Listen I know it’s going to be I’m holding it wrong too, but I’ve been reading white papers and research on LLMs for a long time and was definitely at the cutting edge of context engineering, implementing features in our tooling harness a year before they were in codex or Claude.
maybe I am holding it wrong still but but like at some point if I’m holding it wrong who else will be holding it right? Dozens of people? At some point, the technology has to be approachable enough for everyone to have your point of view automatically.
What do we mean by hallucinate? I'm not counting it making a mistake that it fixes on its own without intervention.
I'm no expert on the inner workings/harnesses/etc beyond a basic understanding of the architecture. Maybe I've just developed a good sense for effective prompts? I could share some recent sessions.
Would you trust it to make life and death decisions? Because it is being used in that context. Drones with AI, armed, and with discretion to choose a target and kill it.
I'm not talking about a hypothetical. I'm talking about Ukraine assymetrically fucking up the Russians with AI drones.
The types of people using it that way are not concerned with the best interests of anyone but themselves, and arguably not even that except within a very immediate time horizon.
Perhaps you should be concerned about it. I certainly am. This is deployed on the battlefield and has been for months.
What makes you think I'm not concerned about it? I've been positively mortified at the seeming cavalier way in which all caution is thrown to the winds. I've honestly had to fundamentally reassess my understanding of the average character of humanity given the last decade.
Fact is, those with the foresight to see how this can go badly are also seemingly the types of people who don't end up in a position to prevent harms by it. Furthermore, it seems inevitable in a sense, because greed for power seems to necessitate development of automated weapons in ASAP in spite of the risks, and the fact it basically renders traditional warfare pointless.
It appears we'll have to learn the hard lessons, same as our forebearers with. I just hope we can avoid having to regress back to sticks and stones on account of fucking ourselves by overdoing our capability to destroy on account of not being willing0able to peacefully coexist.
No, because that's Terminator territory (ULAWS). Since drones' inception, US military required an officer to approve lethal weapon release until at least 2013 and it may or may not still be required. The official policy is DODD 3000.09, which uses the vague word "appropriate" 22 times, but doesn't allow unsupervised lethal autonomous (ULAWS) yet. Maybe someone knows what the criteria are used around in the world's militaries these days beyond Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy (2023-2024) (which Ukraine signed) (There's a competing REAIM 2023 Call to Action that Ukraine didn't sign with a note.[0])
In other news, New Orleans 911 is using AI to triage localized incident duplication calls from unique emergencies. I'm not saying that it's good or their only practical choice, but it's happening.
The biggest dangers I see are the outsourcing of supervisory control, appeal to authority (when used to summarize content or answer a question), and hallucinated mistakes.
0. (PDF) https://docs-library.unoda.org/General_Assembly_First_Commit...
1. PDRMUAIA https://www.state.gov/bureau-of-arms-control-deterrence-and-...
2. REAIM 2023 Call to Action https://www.government.nl/documents/2023/02/16/reaim-2023-ca...
3. REAIM 2023 Endorsing Countries and Territories https://www.government.nl/documents/2023/02/16/reaim-2023-en...
My personal opinion: all output from LLMs is hallucinations. The idea that it’s “right” or “wrong”, when stating factual information is really in the eye of the beholder/user. What year did astronauts land on the moon? — is a question that may or may not have a factual answer depending on your own beliefs. Philosophically, you would need to also define what a fact is, or what “not hallucinating” is, to define what a hallucination is. My understanding is that this property of the LLM architecture is innate, and until we have “world model” LLMs, or some other model that reasons from first principles, instead of the current “guess the next word” model, this isn’t going away. Just don’t rely on the for facts.
I'm talking about software being allowed to make the decision to put bullets through a human being without any human in the chain except the dead guy.
Is this distinct from the hardware in a land mine making the decision to explode shrapnel through a human being without any human in the chain except the dead guy.
Worth noting that both cases indirectly involve the humans that designed devices and the humans that made the placement and trigger condition decisions.
Further:
> AI drones are being used to autonomously target and kill targets by the Ukraine using technology they have been given.
Ukrainian Combat Robot Holds Frontline Position for Six Weeks in Sign of Growing UGV Maturity - https://defenceleaders.com/news/ukrainian-combat-robot-holds...
are remote operated, they allow defenders cover while themselves being out and exposed.
However were they altered to autonomously fire, that would be on the basis of pattern matching in the visible and infra red spectrum - shoot at all hot blobs.
That's more of a trigger threshold setting issue than an LLM hallucination issue, and the danger is on par with any weapon system on auto fire, you really shouldn't approach such things until they are put in a safe off state or have exhausted ammunition.
I guess what I'm really asking is whether or not it's moral and ethical to use AI that targets autonomously? My understanding is that it's being used that way, but I can't back that up and I'll take your word for it if that's not the case. But it is certainly a plausible scenario. Everyone invested in AI seems determined to put it in everything because they need consumers to want to pay for it, and right now they haven't figured out a way to earn back from consumers the over a trillion dollars that has been put into the AI bubble. I don't have much faith that the people with billions invested don't want return on their investment, and I don't think they care about the consequences necessary to get it back.
edit for clarity
> moral and ethical to use AI that targets autonomously?
First point, vision systems have been used in industry to look for misaligned labels, incorrectly folded papers (in high speed paper presses), wrong items on high speed conveyor belts etc. for thirty odd years now - they have issues that a very distinct from LLM 'AI' hallucinations.
That's nomenclature out the way.
Landmines are indiscriminate, they trigger on any weight or pressure over a threshold.
A vision based Felixer, by contrast, only triggers on cats (well, almost always only) and leaves bilbies and bettongs to walk on by.
That's an improvement over landmines.
The crux of your issue here might be the morality and ethics of establishing human exclusion zones within which all humans are highly likely to die.
These historically are created with rapid patterned artillery fire, butterfly mines, Napalm, indiscriminate criss crossing machine gun fire, etc.
Now there exists an option to use drones to kill all humans and leave the horses and cows alive.
Is it your concern that a bad vision threshold might kill a horse rather than a person? (Likely not)
Would you prefer an area to be napalm'd and agent orange'd back to dust?
War is hell.
* https://www.abc.net.au/news/2020-05-29/feral-cat-management-...
Also, why do people keep talking about landmines? I'm talking about software, you are talking about dumb mechanical things from WWII. This isn't a philosophical question about methods of warfare from the past, but the software that controls the dumb hardware.
> why do people keep talking about landmines?
I, a person, singular, mention landmines as they are a hardware device that are designed to trigger action on a threshold.
Felixers are computer vision based devices that are designed to trigger action on a threshold.
These are actual real world objects that Bishop Berkeley can kick, not vague philosophical questions but actual engineered devices that kill and are in use today.
I'd prefer a world without war, rather than one that is starting to make the movie Terminator 2: Judgement Day a reality.
yes, yes it absfuckinglutely is. IMHO