I've been researching AI detection (and also made a game to see if humans can quickly detect AI writing just based on style in a short passage: https://prose-or-con.com/ as a way to gather data and to sate my own curiosity about whether I can actually detect AI writing reliably...I can, but not as reliably as I thought).
Storyscope (https://github.com/jenna-russell/storyscope) is a really interesting concept, and one I'm in the process of reproducing and expanding on, even though it is less accurate on direct detection than something like BERT or Binoculars. It is interesting because the usual obfuscation tactics, even having a human transcribe and rephrase the story don't work; it looks at the story itself, rather than word use and grammatical quirks, etc. Plot, agents, temporal structure.
So far, AI writing is detectable to a pretty high degree, though I think it'll be a war of attrition that AI eventually wins. And, all the detectors are the same tools one could use to create undetectable AI writing; AI loves to iterate in a loop, seems like iterating to rewrite to be undetectable is a soluble problem (though models currently probably would end up writing worse and worse to avoid detection, and the current best models have begun integrating watermarking, pushing back the defeat of human writers for some time).
This was an interesting test, thank you for posting the link. I also found the AI writing in it harder to spot than I expected, but I suppose I'm mostly tuned in to the low-quality Claude-style slop that's most commonly posted to HN. The coding-oriented models seem to be optimised for feedback-driven code creation at the expense of the writing style (which stinks and is quite easy to spot).
When instructed to emulate a style (like "travel writing" or "an encyclopedia style article") the best models are very difficult to detect in short passages. Longer work does tend to out even the best ones, though. But, I couldn't make a game of "read these ten pages ten times" and expect anyone to play it. Even now, it's not exactly a rip roaring good time, more of an interesting experiment that curious folks do once or twice just to find out how well-calibrated their built-in AI detector is.
I'd like to come up with a more fun game loop, but nothing has revealed itself to me yet. Shorter passages are easier to gamify, but shorter passages are much harder to detect.
I think AI detection will increasingly be an arms race .. so we might need other solutions, eg :
- blockchain attestation [ crypto-graphically signed by human author / artist ]
- proof of work in the traditional sense, eg video of artist at work, stages of the painting from sketch to under-painting, brushstrokes, layers, glazing
yes, Im aware the second can be increasingly faked .. eg train a DNN or LLM to mimic brushstrokes and oil paint handling
In my case, I think videos of the painting process are a reasonable, if temporary, solution.
> I see nothing wrong with readers gobbling up AI-written books for the rest of their lives.
> Because to me, authenticity makes for a better reading experience.
This pair of sentences gave me pause, until I realised that they're not in contradiction - the author is deliberately non-judgemental of _others_ making a personal choice that _he himself_ would prefer not to make. Remarkably rare, nowadays.
They are in contradiction, because the author starts by saying how much he enjoyed the Stratemeyer Content Mill books (full disclosure - I did too) and then says that if only he'd known...
But he didn't. And it didn't affect his experience until he found out.
Which strongly suggests there was nothing wrong with the reading experience, and no lack of "authenticity" in the text itself, on its own terms.
It was a chimera.
As far as most readers knew, Carolyn Keene, Franklin W. Dixon, and Victor Appleton were all authentically real people.
Discovering they weren't didn't change the quality of the writing - it only changed the implied relationship with the author.
But that's a parasocial experience, not a feature of the words on the page.
I suppose the implication is that because the book went through the mill the author and publisher somehow didn't really mean them.
Perhaps they were tainted by money instead of being products of pure enlightened generous impulse.
But how can you not mean something like a Tom Swift book?
The series was incredibly successful and had real social influence. Would it have been more successful if Victor Appleton wasn't a made-up person?
Why do we expect authors to be scrupulously physical, when creating imaginary people and imaginary worlds is literally the job description?
While I absolutely agree with your points, and do feel the same towards books I've read and later revisited similar to the ones alluded to. I would echo the transparency of giving readers the choice rather than the thought process that it'll still be enjoyable, therefore the deception is fine.
Especially as I feel that more and more this degree of transparency will be crucial for people to make more informed decisions in what they wish to read - and by the author's point, where do they want to spend their money.
What if an individual not knowing any better could absolutely enjoy AI written stories from cradle to grave, but would rather live in the blissful innocence of contributing to a lone writers' income rather than content farms? Isn't that enough reason to want transparency here?
My point being, if there's really "no harm" in the deception of either human-powered or AI-powered content farms. Then let's opt for imposing the transparency for readers to truly have the choice.
This is a remarkably magnanimous and even-tempered post. Most authors (e.g. Charlie Stross) really hate AI, feel personally affronted by it, and lose their cool when they talk about it.
I agree with Howey's sentiments. AI-written books aren't bad in themselves -- they're not ontologically bad -- but they tend to be, by their very nature, derivative. Narrative quality is, as yet, still quite poor. And self-publishing on Amazon.com is full of that stuff. (I've long since stopped buying self-published stuff on Amazon on account of low-quality content, and Amazon's review system seems severely broken.)
Humazon would be a welcome thing. Or even a "human written" flag for books on Amazon.
"So here’s what I would love: A bookstore for humans by humans. A Humazon.com. Or even just a BookBud recommendation service where the authors are fully vetted (they’re your buds!)."
--------------
To do what the author of this post asks for is incredibly difficult. You can't reliably use a LLM to detect the output of other LLM's. You can't trust user reviews to distinguish AI from human writers because AI can easily produce reviews. You'd basically need to form a central gate-keeping authority that decides who is allowed and who isn't. This would be resource intensive and fraught with problems. (e.g. How many authors would sue if they were blocked as the result of a false positive? How many using LLM's to write their material would brazenly sue for being blocked too?)
It really bothers me that one of the biggest problems with LLM's is how difficult it is to filter out what they produce.
I've been researching AI detection (and also made a game to see if humans can quickly detect AI writing just based on style in a short passage: https://prose-or-con.com/ as a way to gather data and to sate my own curiosity about whether I can actually detect AI writing reliably...I can, but not as reliably as I thought).
Storyscope (https://github.com/jenna-russell/storyscope) is a really interesting concept, and one I'm in the process of reproducing and expanding on, even though it is less accurate on direct detection than something like BERT or Binoculars. It is interesting because the usual obfuscation tactics, even having a human transcribe and rephrase the story don't work; it looks at the story itself, rather than word use and grammatical quirks, etc. Plot, agents, temporal structure.
So far, AI writing is detectable to a pretty high degree, though I think it'll be a war of attrition that AI eventually wins. And, all the detectors are the same tools one could use to create undetectable AI writing; AI loves to iterate in a loop, seems like iterating to rewrite to be undetectable is a soluble problem (though models currently probably would end up writing worse and worse to avoid detection, and the current best models have begun integrating watermarking, pushing back the defeat of human writers for some time).
There is also this, Wikipedia's AI or not quiz:
https://en.wikipedia.org/wiki/Wikipedia:AI_or_not_quiz
This was an interesting test, thank you for posting the link. I also found the AI writing in it harder to spot than I expected, but I suppose I'm mostly tuned in to the low-quality Claude-style slop that's most commonly posted to HN. The coding-oriented models seem to be optimised for feedback-driven code creation at the expense of the writing style (which stinks and is quite easy to spot).
When instructed to emulate a style (like "travel writing" or "an encyclopedia style article") the best models are very difficult to detect in short passages. Longer work does tend to out even the best ones, though. But, I couldn't make a game of "read these ten pages ten times" and expect anyone to play it. Even now, it's not exactly a rip roaring good time, more of an interesting experiment that curious folks do once or twice just to find out how well-calibrated their built-in AI detector is.
I'd like to come up with a more fun game loop, but nothing has revealed itself to me yet. Shorter passages are easier to gamify, but shorter passages are much harder to detect.
I think AI detection will increasingly be an arms race .. so we might need other solutions, eg :
yes, Im aware the second can be increasingly faked .. eg train a DNN or LLM to mimic brushstrokes and oil paint handlingIn my case, I think videos of the painting process are a reasonable, if temporary, solution.
> I see nothing wrong with readers gobbling up AI-written books for the rest of their lives.
> Because to me, authenticity makes for a better reading experience.
This pair of sentences gave me pause, until I realised that they're not in contradiction - the author is deliberately non-judgemental of _others_ making a personal choice that _he himself_ would prefer not to make. Remarkably rare, nowadays.
They are in contradiction, because the author starts by saying how much he enjoyed the Stratemeyer Content Mill books (full disclosure - I did too) and then says that if only he'd known...
But he didn't. And it didn't affect his experience until he found out.
Which strongly suggests there was nothing wrong with the reading experience, and no lack of "authenticity" in the text itself, on its own terms.
It was a chimera.
As far as most readers knew, Carolyn Keene, Franklin W. Dixon, and Victor Appleton were all authentically real people.
Discovering they weren't didn't change the quality of the writing - it only changed the implied relationship with the author.
But that's a parasocial experience, not a feature of the words on the page.
I suppose the implication is that because the book went through the mill the author and publisher somehow didn't really mean them.
Perhaps they were tainted by money instead of being products of pure enlightened generous impulse.
But how can you not mean something like a Tom Swift book?
The series was incredibly successful and had real social influence. Would it have been more successful if Victor Appleton wasn't a made-up person?
Why do we expect authors to be scrupulously physical, when creating imaginary people and imaginary worlds is literally the job description?
While I absolutely agree with your points, and do feel the same towards books I've read and later revisited similar to the ones alluded to. I would echo the transparency of giving readers the choice rather than the thought process that it'll still be enjoyable, therefore the deception is fine.
Especially as I feel that more and more this degree of transparency will be crucial for people to make more informed decisions in what they wish to read - and by the author's point, where do they want to spend their money.
What if an individual not knowing any better could absolutely enjoy AI written stories from cradle to grave, but would rather live in the blissful innocence of contributing to a lone writers' income rather than content farms? Isn't that enough reason to want transparency here?
My point being, if there's really "no harm" in the deception of either human-powered or AI-powered content farms. Then let's opt for imposing the transparency for readers to truly have the choice.
This is a remarkably magnanimous and even-tempered post. Most authors (e.g. Charlie Stross) really hate AI, feel personally affronted by it, and lose their cool when they talk about it.
I agree with Howey's sentiments. AI-written books aren't bad in themselves -- they're not ontologically bad -- but they tend to be, by their very nature, derivative. Narrative quality is, as yet, still quite poor. And self-publishing on Amazon.com is full of that stuff. (I've long since stopped buying self-published stuff on Amazon on account of low-quality content, and Amazon's review system seems severely broken.)
Humazon would be a welcome thing. Or even a "human written" flag for books on Amazon.
"So here’s what I would love: A bookstore for humans by humans. A Humazon.com. Or even just a BookBud recommendation service where the authors are fully vetted (they’re your buds!)."
--------------
To do what the author of this post asks for is incredibly difficult. You can't reliably use a LLM to detect the output of other LLM's. You can't trust user reviews to distinguish AI from human writers because AI can easily produce reviews. You'd basically need to form a central gate-keeping authority that decides who is allowed and who isn't. This would be resource intensive and fraught with problems. (e.g. How many authors would sue if they were blocked as the result of a false positive? How many using LLM's to write their material would brazenly sue for being blocked too?)
It really bothers me that one of the biggest problems with LLM's is how difficult it is to filter out what they produce.
This comment was AI-generated, wasn't it?
Turtles all the way down I swear.
I can deny it, but not disprove it. That's pretty F'd too.
see also, his : https://hughhowey.com/the-end-of-an-era/
[flagged]