As someone who is aggressively pro AI, I've also noticed how awful everything is to read lately. It's always so hard to put my finger on the exact thing, but it's a mixture of everything sounding identical and everything being a rambling unconcise mess. I find I routinely start reading interesting posts and just .. stop part way, abandoning it. A site I read stories on has been flooded with AI generated content, all of it terrible, and it just makes me retreat to things created before 2023 as a way of filtering it all out.
A model trained to minimize statistical prediction errors is going to sound ... predictable (aka boring). A (good) human author tries to keep their writing interesting, while an LLM strives for predictable instead.
It's not just predicting content but also the form/shape of the response, highly shaped by RLHF preferences, which seems to encourage longer form responses which need padding with filler to generate the "preferred" amount of verbiage.
It seems most of these models have been trained to interpret questions as wanting a long-form authoritative/comprehensive-sounding response (even when midwit filler is all it has to offer), rather than initiating a short-turn conversation which would better allow you to steer the conversation. Apparently Muse is much better in this regard, but so far I've resisted signing up for any Meta/FaceBook service, and not inclined to break this winning streak!
Human writers also use specific constructions for definite (though not always consciously known) reasons, and mapping those correlations onto writing models is only getting done on accident. This is a big part of why LLMs doing pointless negations (it's not X, it's Y) is so jarring: people use that construction in order to preempt an expected misconception, whereas LLMs try to use linear algebra over the learning corpus to generate it in appropriate quantities and places according to RLHF/fine-tuning/etc.
In other words, it isn't just verbiage that has a "preferred" quantity, but also higher-level language features, and the appropriate contexts for those features feel like the sort of thing that will be extraordinarily difficult to replicate without a theory of mind.
edit after coming back and re-reading: the "in other words," and rephrase is because I'm trying to explain something complex enough that I'm not even sure I fully understand it, so I try typing out multiple explanations and leave the ones that make the most sense to me. This might be idiosyncratic to my writing and not a significant feature of training corpuses, but even if it wasn't it's a second example of the sort of thing I'd suspect LLMs to be bad at using appropriately.
Another LLM weirdness is to explain what code is not doing. My theory: such negations were somehow encouraged in training because they are very valuable where used properly (and sparely) by humans. "Not doing the obvious thing due to xyz happening two modules over" is a very useful comment. "Not using bubblesort because quicksort is faster" is garbage.
I think a lot of the time it's prompt leakage. Forgive the personification, but it's like they don't have a mental model of the reader of the thing that they're authoring being separate from the person who's prompting them to author it, so it often seems like the main conversation kind of spills into the prose.
Not to pick on GenHTTP Lambda, especially because I think it's someone's fun side project, but I was just trying to read though their docs and I was finding it super difficult. I kept getting distracted thinking "why the hell did it write it _that_ way?"
Like, the first sentence of "The control center" section [1] is the exact sort of weird negation that you're describing:
> The editor link opens a control center rather than a text box: most of the code here is written by agents, so the first thing on the screen is how your lambda is doing.
I can't help but think it generated the first round of the page with a text box to paste the code into, got told "no, I want a control center," and then this ended up in the docs.
I also think the phrasing of that sentence (and many others in those docs) is just super weird and unnatural - e.g. "so the first thing on the screen is how your lambda is doing." It also feels like it mirrors a prompt - like, "no, it should be a control center, not a text box, and the first thing the user should see is how their lambda is doing." But in docs I'd never expect it to be written "the status of your lambda" or "the health of your lambda." And what does the code being written by agents have to do with it?
Lambda is my side project and I 100% agree. Updating documentation with Claude does not work, unless you spend so much effort that you could have written a perfect version yourself. Opus 5.5 got better at it, but it's still a long way to go. I personally cannot stand the PR descriptions generated by Claude. Three pages of text for just a simple change ...
This is my experience as well. I have spent many hours trying to eliminate this kind of contamination and the only solution I've found is to re-write from scratch every time. If the result isn't to your liking just reset its context and provide a more detailed prompt.
I vaguely remember a study that I was able to find when fact-checking this comment before publishing it [1], which found that music needs periods of high predictability to nurture listener expectations for the sake of later subverting those expectations, and the most enjoyable parts of music are those moments where expectations are subverted. This sounds like a similar effect.
Yes, with the "scenario expectations" just being the necessary setup to deliberately create false expectations and make the punchline that much more unexpected/funny.
I think both music and humor are both exploiting the same evolutionary essentials of human nature - to find comfort/safety in predictability, and excitement/danger in unpredictability. Laughter would seem to be a social thing - perhaps it helps develop a theory of mind in signalling that you get the joke - that the punchline is indeed unexpected to you as well ? And/or perhaps it helps draw others attention to an unexpected situation with learning potential.
Agreed. In my experience it gets to the heart of information faster rather than so much filler. Also better information and more comprehensive in general.
The main issues seem to be glazing that sibling mentions and also it seems to play a bit loser with facts and reality and makes up more stuff.
Every model has done this. And sometimes the makers go the other way to compensate, and then you get Opus 5 who answered like it was possessed by the ghost of stack overflow.
Not really, Gemini is at a different level, it will naturally burst out compliments and honorable nickname to provide emptional values. My last company was iterally destroyed because the old boss (successful boomer who had no idea how software industry work) believe everything his Gemini told him and gives us all kinds of unrealistic assignments and force us to use whatever tools/services Gemini suggested.
"A model trained to minimize statistical prediction errors is going to sound ... predictable (aka boring)." -- what if the goal is to be a (good) human author.
In that case, model will in fact strive to keep their writing as close to interesting writing as possible.
The goal (literal, not figurative) of an LLM is to reduce training/inference errors - to predict the training material.
The "quality/goodness" (likability) of output is only selected for to extent that it also reflects RLHF preferences originating from low paid contractors who are scoring based on "looks good to me", not based on Pulitzer prize prose potential.
To explore a false hypothetical, suppose an LLM was only trained on the output of a single popular author, then you'd certainly expect it to reproduce that author's style, but it would not be "upredictably interesting" but rather predictably quirky, reproducing the same "surprises" over and over so that they no longer are surprising and therefore interesting.
To continue the hypothetical, if you trained an LLM exclusively on the output of multiple good authors, then rather than get the best of each you'd get a statistical blend (this is how training works) that emphasized what they had in common rather than the individual quirks that made them interesting.
There is a fairly common misperception that "AI" (LLMs) is always trying to do it's best, where "best" has been anthropomorphized to have the human sense rather than the LLM sense of "the most statistically accurate continuation prediction" it can make. For example, if you give an LLM the moves of a partial chess game and ask it to predict next moves for one of the players, it will NOT play to win, but will rather play to predict how the game would have continued, which means continuing at the level (ELO rating) and style of the player who had made those moves.
Of course you can ask an LLM to play to win, just as you can ask it to write in a style that is, say, "unpredictable and interesting", but it can't escape itself - you will necessarily get predictable unpredictability.
I think you're missing what I'm saying; the LLM can and will have trained on all interesting prose, and it will evaluate all interesting prose. If you indicate that you are goal setting for amazing prose, and you give enough chain of thought, it will self evaluate.
Your chess game example misses the point. If you give an LLM the moves of a partial chess game and ask it to predict the moves of the players, it will predict how the game would have continued, even if it is low level players. But if you ask it how Magnus Carlsen would win the game, it will likely come up with a sequence that can beat most human players, or possibly just decide to invoke a chess solver, which will most definitely win.
The key for your prompts and thinking loops is you need to indicate proper goals. "Write the a story about 1775 England." is different then setting a goal for it to write the a fantastic storyin the style of Dickens about England in 1775.
Regarding the ML portion; what you've written about a 'statistical blend' is a little bit coarse. That would be what you might have gotten from a 2 layer neural net or a regression in 1999; the neural networks today operating with attention or an incredibly high number of parameters are non-linear and non-interpretable, that's the entire point. This isn't a regression of simple parameters, it's all interactions of all parameters deeply netted. Model interpretability vs. Model Prediction power are generally inversely related.
The model can in fact escape itself in that it has many, many, many more parameters than a simple "predictable unpredictability" and that is exactly the point of it.
I'm not sure how you are imagining an LLM "self evaluating". They are feed-forward auto-regressive predictors. The only source for the next word they spit out is what came before and the patterns that contains.
If you ask an LLM to "win like Magnus" it can't because, as above, it is a feed-forward auto-regressive predictor. Do you realize how much different this is to how something like Stockfish (that can play at, and far beyond, Magnus' level) works?
A big LLM is still an LLM - the architecture and operation doesn't change. It just has more capacity to learn, both to learn deep predictive patterns and to memorize factual knowledge.
Does it not occur to you that your beliefs of what LLMs are capable of (generating amazing prose) is totally at odds with what we see them doing? Where is all the amazing prose?
I'd be shocked if it wasn't a shotgun strategy learned during RLHF or RLVR post-training.
A short response that's wrong is going to be punished, whereas a long answer that contains the correct information, and then a whole bunch of other stuff surrounding that info, is more likely to be rewarded. Even if you discount the reward by answer length, when there's ambiguity involved it's still going to be a better strategy on average to hedge by saying too much vs risking saying too little.
That doesn't follow. Predictability is your brain reacting to the lack of novelty. What an LLM produces is a chain of words following some kind of conditional distribution, whose output could vary significantly from your understanding of novelty depending on its training and priors.
With a single, generalized prompt ("produce something unpredictable"), ChatGPT responded with:
> 00:— / ANOMALOUS RESIDUE
> Vekh.
> Ìssara.
> Nnnn—tq—èl.
>
> A temporary thickness enters the meat of the signal.
>
> Not rain. Not a substance with permission to fall.
>
> UL / 9 / ferrous / ∂
> The measurements are eating each other.
>
> Someone has left an entire childhood in the wrong unit of pressure.
That is not predictable to me, at least. It could be boring, but I think that is mainly a product of me knowing where the text is from. LLM-generated text is inherently boring, to me, but that has nothing to do with predictability.
Don't these sort of conflict with being "aggressively pro AI"?
> how awful everything is to read lately
> flooded with AI generated content, all of it terrible
> everything sounding identical and everything being a rambling unconcise mess.
I can see the benefits of AI and enjoy using the technology - but, like many have said, it's wreaking havoc on the "content" that's out there nowadays. Gotta take some bad with the good, I suppose.
Well, it's quite simple if you think about it. Intelligence is all about awareness of the imminence of death and the finite amount of time given to any of us. So, we humans intrinsically evolved to make the most of it. The majority of the tools we create have the ultimate purpose - to save time. Even money is nothing but a way to cut some time. It's not really about time but experiences - we want to have as many and as rich as we can get, and we innately try to cut time somewhere else, so we can use what we saved for better experiences.
Therefore, inventing cars was an easily predictable event. There's not a single hypothetical timeline or dimension where humans don't end up with cars - to transport things and to travel faster. Similarly, it was always simply inevitable - we would have invented something like LLMs, sooner or later.
Cars required significant sacrifices - they are absolutely deadly machines, they kill thousands of people every single day, kill wildlife, endager our own existence, and we're kind of okay with that. I can absolutely assure you, if someone tomorrow invents a gravitational jump travel that allows moving things between continents in minutes, even with a success rate as low as 10%, we would still be using it.
So, yes. We can't un-invent cars and AI, the only way to move forward is to adjust to this new reality and extract the best possible experiences, preferably by not robbing other people of their - i.e., enjoy life while it lasts, it still remains finite.
> Intelligence is all about awareness of the imminence of death and the finite amount of time given to any of us.
You lost me there. Children are pretty intelligent and have no concept of time. I don't see the relation you find obvious and the rest of your points similarly don't follow.
> Therefore, inventing cars was an easily predictable event.
This feels like a benefit of hindsight unless you can call out other obvious and imminent (but not yet existing) inventions.
Yeah, I guess, "intelligence is all about awareness of death" is kind of a bomb of a premise. I admit, not the best thought I spat out through my keyboard. I suppose I mean some narrower sense of intelligence here. On one hand, you're absolutely right - crows, octopuses and toddlers solve problems without any hint of understanding mortality. Yet "children have no concept of time" is incorrect: not only do they have one, but most kids also grasp that death is irreversible and happens to everyone by about age 5-7.
I don't blame you for immediately rejecting the rest of it. But do hear me out: the point about the inevitability of invention does have real support. Historians call it multiple discovery - tons of things have been invented independently - calculus, the telephone, the jet engine, to name a few.
Neural nets, GPUs, and the scale of Internet text made something like LLMs close to inevitable. Of course, we could debate forever whether any of these on its own was ever inevitable. The point is - society tolerates any cost of a useful invention for the benefit of saving time. So it's reasonable to assume this will be the case with AI as well, which is really happening as we speak.
Ironically, this is the case where "nothing is better than more", because I have no idea of what property or context of any of these three comments above you're referring to.
> Don't these sort of conflict with being "aggressively pro AI"?
I don't think so. AI is broad, and he's against low quality output enabled by people taking shortcuts using AI.
Using AI doesn't make your output worse, that's almost entirely down to lazyness and a lack of skill. Or people are just choosing to spend less time on the same piece of work instead of doing additional iteration and experimentation (which AI makes a lot more doable).
I can like the concept of the engine, and the leverage it provides, but I hate traffic and get anxious about having an accident on the freeway. And the people behind the industrial revolution have destroyed the climate.
I love computers and the internet, and the internet has ruined a lot of people and things around me.
It's possible to be interested in a new technology and enjoy its leverage while still being extremely apprehensive about how it intersects with game theory and politics. The race to the bottom of the barrel keeps coming for the jugular of every cool new thing.
I’ve been starting to think that a lot of the variance in how people see the world can be explained by whether their attention is drawn to first-order localized effects, or nth-order distributed effects. This is not the same thing as optimism vs. pessimism, but it sure is correlated!
It's a multi-mandate to focus on what's best for your yourself and society such as your mind, your body, your chosen family, your friends, your neighborhood/community, your city, state, country, the global population, and more abstract things such as the human condition, when evaluating things such as privilege or fairness or duty.
You often have to make compromises somewhere in the hierarchy, especially in late-stage capitalism where even access to basic needs comes at a moral cost, and as such those compromises sometimes come with a moral duty. I am writing this via a computer built from limited natural resources, some of which have been obtained through what I see as slave labor. I don't eat meat and still have to acknowledge the harm that clearing and maintaining farmland does to wildlife, even if the result could theoretically be a net gain.
When you start viewing situations through the lens of game theory, you realise some compromises are inevitable for the rational individual. But you must acknowledge the individual is not the only concern. As such, I use AI as part of my work, but I opposed the local buildout of a large data center in my community and tried in vain to make the person responsible for the deal understand the issues it could introduce. This isn't NIMBYism, we need a halt on all buildouts until fundamental environmental and accountability concerns are addressed.
And many other aspects of my existence and quality of life, poor and underprivileged as I was growing up, have depended upon US imperialism. I have also produced a ton of waste just existing and owning pets. It's abhorrent.
I continue to ask myself how I can fulfill the moral duty which I have accrued and reduce my dependence on exploitative economic systems, but I also recognize that AI is the new social leverage and we only have so much time to use it to our advantage before it becomes institutionalized and wielded against the people. I also might only find future income stability by evolving with my industry.
Well, we have a good understanding of the benefits that the industrial revolution and the computing revolution have given us.
The AI situation is different. The technology is obviously extraordinarily powerful but the benefits to humanity are still nebulous and there are many very obvious harms like the one we're discussing. So to describe yourself as "aggressively pro AI" you must believe that future, as-yet unrealized benefits will outweigh the harms. I don't think that's a crazy thing to believe but it deserves scrutiny and demands elaboration.
These challenges never go anywhere because it’s essentially a religious conviction. We’re likely in sort of uncharted territory on a macro scale and I reckon people a) either acknowledge that or refuse, and b), either trust or distrust the tech world’s stewardship of this technology. There’s a hundred bazillion data points, and I’m pretty pessimistic about the tech scene at large right now, but I think reasonable people can disagree on this without being factually or logically wrong. But true zealots in the midst of such uncertainty are driven by faith, which can certainly be deftly swayed over time, but definitely can’t be conquered in a debate.
Eh, I think many of the harms of AI are currently also nebulous. It's not as though the harms are all concrete and massive and the benefits are vague and "in the future"
I agree but I see a lot of very concrete harms too. It's been catastrophic for schools, the sloppification of the web has been largely a disaster so far, it's been a disaster for cybersecurity, it's wreaked havoc on the consumer electronics market, it's driven a lot of financial and existential insecurity for students and early career people across various roles, and while I'm not as militant about data centers as some I think their proliferation has been a net negative environmentally (including increased reliance on coal).
These are ultimately all software systems where updates can be pushed out with the press of a button. We just found an amazing new tool in the LLM and people are trying out all kinds of ways of using it, many of them dumb or stupid, this is discovery that has to be done, we'll keep doing it and pushing updates.
It doesn't take rose colored glasses really to infer that over time we'll cull the dumbest uses and extrapolate on the best ones. There's a huge difference between using a generic chatbot, even one driven by a SOTA model, vs targeting a specific business task with good guardrails, skills, and tool interfaces that call out to real business logic when the model isn't the right thing. The latter will emerge more and more over time.
You can be pro-"thinking AI is going to get really good and solve many problems" (i.e. be the sort of person who would "go long" on AI), while also being anti-"applying AI right now to domains outside of the few it's been specifically optimized for [e.g. coding]" (because in these other domains, AI is yet to approach, let alone surpass, "authentic quality" equivalent to that of a skilled junior professional.)
Only a programmer or a business guy would think just because it becomes capable we would want to see AI produce "content". The effect is the erasure of human culture.
This is somewhat of a dramatic statement. What, exactly, is the mechanism by which human culture is erased because LLMs write some stories or compose some music?
The most powerful people in silicon valley are openly aiming to replace the entire species with robots in space. This is their genocidal aim. The fact that bad things haven't happened yet doesn't mean that directionally, they aren't attempting to replace as much human culture as possible.
It sounds silly! That's because they are completely divorced from the human experience, and they have enough power to really make us all suffer.
I think there could be room for something like how holodeck programs are shown to be authored in Star Trek, i.e.: the author does all the primary work, and then the system, if a particular work's design allows it, extrapolates from that to fill in the blanks when somebody decides to make Moriarty super-intelligent or put a hundred androids in a Wild West scenario or take part in a go-karting death race with Frankenstein's monster.
But, that comes with with the same kind of cultural acknowledgement, as seen with the characters in Star Trek, that such algorithmically generated material is an experimental diversion, and inherently not worth the same respect or focus as experiencing something authored with intentionality by a person.
Right, deployed thoughtfully, in a way that improves the human condition (especially aboard a ship far from civilization!) there are uses of course. When everyone is offloading their thinking and writing this is effectively erasing the production of culture.
I'm sometimes those things. I'm more often a writer and a futurist. I am enough of a writer specifically, to really not want to live in a world where AI-generated prose substitutes for human stories and essays; and I feel empathy toward other creative professions (most of my friends are visual artists) and don't wish to see that kind of thing happen there either. I do think we have to push back on things like that happening. And I do think that people trying to push toward that end for monetary gain are erasing much of the joy of being human.
But that being said. I don't think that our need to push back on that particular use of AI, has anything to do with the need to push back on AI itself. Because that's not the only, or to me even the main, point of (generative) AI.
(It's just that "generating wholesale slop that people end up seeing" is the usage-profile that the AI companies can most-easily intermediate and make money on; and it's also the usage-profile that makes it most obvious that "AI is happening", and so markets AI to people not already using AI. So this is the usage profile that all the AI companies benefit from everyone focusing the conversaton around AI on!)
I think a lot of people have a failure of imagination when they think about the upside of "good" generative AI. Which to me is, specifically, the places where generative AI can be better put to use as a tool in a workflow for human creative labor to pass through, rather than being something that substitutes for said human labor.
The programming example might be the most obvious to people here: generative AI lets you write e.g. a hairy SQL statement in the form of a very long natural-language description of the query, or an example of how you'd do it in an actual imperative/functional programming language, and have that transformed into SQL grammar. So you don't have to try (and fail) to remember the arcana of SQL (e.g. how to use things like correlated subqueries, or window functions with explicit windows, or CUBE), always eventually resorting to looking it up anyway. (This is really a condemnation of SQL for having specific one-off syntax sugar for every single thing it wants to encode, rather than a uniform syntax that can fit in your head / trigger IDE auto-suggest.)
But consider Photoshop's "generative fill" tool. Yes, if you're selecting entire huge regions of the image and running that tool on them, it's going to be "doing original work" and the results will be awful. But that's not the use-case the tool was designed for. It was designed to be a better version of content-aware fill, i.e. PatchMatch. If you select a dust speck or tear on a scan of an old photo and run the generative-fill tool over it, the result will usually just be a better version of what you'd have got by running the content-aware fill tool over it. (And if it's not, you don't need to try to fight the tool; you just drop back to using the other tools in the toolbox. Not every tool is applicable in every situation, and that's okay.)
Or consider that newer versions of Melodyne use generative-AI models to better separate audio stems into their own tracks. The input here is your own manually-performed track. The output here is also your own track. Despite being a generative model, the AI here is only generating an internal byproduct that gets discarded (hypotheses for timbre- and acoustic-matched reference instrument performances) to aid the classifier and reduce stemming error/residue.
(And apparently, the compressed-sensing models used by MRI scanners are getting approximately the same benefit by approximately the same means. Less familiar with this, but it's probably the most important benefit here since improving the fidelity of MRI reconstruction means doctors can be more sure that an opacity is really "something" rather than an artifact, and so will have more confidence to diagnose real problems earlier.)
Or consider generative AI used for automatic human-language translation (where it captures much more of the speaker's original meaning and nuance than previous translation models, at the expense of giving the result "LLM voice.") As AI improves at this task, presumably there will be less "LLM voice" and the result will just sound like what the person themselves would have said if they were bilingual, because... that's the task.
In none of these cases is generative AI "producing content." In these cases, generative AI is subsuming the role not of human labor, but of tools that were already used: of "by-the-book" approaches, of heuristic algorithms, of weaker ML models relied upon by heuristic algorithms, etc. But all of these cases benefit from generative AI being improved.
And, to restate my premise: there are many tasks just like these, where generative AI currently has no place (because replacing or "enhancing" any tool in the task's workflow with gAI would currently make the results worse), where you can expect to see gAI eventually replace or "enhance" some tool in the workflow, once gAI model companies turn their attention and benchmarks to trying to optimize for performance in those sorts of tasks.
Agree but also want to point out that this started with the SEO boom of the last decade where sites started rambling so they would keep you on the site longer and get more SEO "points", they were articles meant for a robot to catch humans.
AI has made it even worse and lots of content on the web, for both these reasons, is not worth you time (now wonder users don't read most of the text on our apps!)
In a prior evolution of this, if you look into the Get Rich Quick marketing copy of the early 2000s (before SEO had taken over content formatting), you can see a parallel of sorts in those marketers doing human sales optimization Impact with bold and underline. It derives from research into the human response to ads from decades of print ads and centuries of printed matter prior, and I think there is a direct lineage from ‘how can we bias humans through restructuring text’ to ‘how can we bias machines through restructured text’. This isn’t to disagree, but to call out how rapidly humanity (merely twenty years!) has switched from marketing to people, to marketing to algorithms, in search of a favorable bias.
I suspect it’s partially an average / regression to mean.
If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.
When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.
The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.
It’s heavily RL’d and it’s gotten worse because they’re using LLM judges, that’s why it’s gotten worse. The last good writing model was GPT 4.5.
I suspect they’re leaning heavily on some kind of auto verifier, and that’s why the writing sucks, because good writing is so subjective.
And they’re probably nerfing what it can get from training as they’ve lost lawsuits for scraping copyrighted work, so they’ll purposefully avoid anything that references a specific writers style, etc.
Isn't this easy for frontier 'labs' to fix? If the user requests lots of writing and prose, then use an auxiliary model which has been trained on 'award-winning' writing instead of all the crap on the internet?
Award-winning writing has all kinds of different styles. The average of them will probably get you some boring median soup once again, though maybe a bit less bland than the LLMs trained on oceans of LinkedIn and Reddit slop.
Even if it doesn't, the effect of so much writing getting pushed out the door through default LLM settings is going to make it feel same-y and bland in short order.
The "regression to the mean" or "it's just the training data" hypothesis doesn't work. Pretrained models do not have these weird writing styles and sound much more human. It's only after instruction tuning -- SFT, DPO, etc. -- that LLMs develop weird writing styles.
That is, the original training on massive text corpora seems to produce an average of human writing, but that average looks pretty average. It doesn't feel like LLM slop. But the instruction tuning on specific tasks (answer this prompt, summarize this text, do this math problem), which turns text predictors into chatbots, seems to create strange style features as a side effect. And because that tuning step depends on company-specific training data (rather than just a crawl of the entire Internet), each company's LLM has a different style.
It's not that difficult to measure this. Linguists have had measures of style variation for a long time, though NLP and AI researchers seem to have forgotten. We've generated parallel human and AI corpora to measure it, and can show specifically which writing features change: https://arxiv.org/abs/2410.16107
Currently we're working on tweaking the instruction tuning process to see if we can pin down which steps and what types of tasks tend to produce the weird style features.
I didn’t really specify whether the regression to the mean was from data or tuning. It’s just happening. They are creating neutral average language instead of varied expressive.
The part I find grating is the attempts to punch everything up. ("It's not X. It's Y"). Like, its a react badge component, it doesn't need punching up. You didn't find the smoking gun in a murder mystery, just a misplaced CSS rule.
I've noticed this as well, I've been comparing it to 4Chan Greentext memes. Everything is a short punchy sentence, and every sentence has the same "weight" to it. There's no flow or rhythm, it's like reading off bullet points.
I call it uncanny valley effect for text [Ed: ops use of prose, rather than text is better formulated]. It's a little like with AI images - they started out with six fingers, but now they look "ok" - but slightly off.
Reading some AI text feels like looking at a Michael Murphy sculpture - makes perfect sense, until you start to look closely at the text, and gaps appear - and you start to realize there's no there, there.
My model is about gaps - all ideas start with gaps. LLMs can fill in these gaps quickly - but some ideas are just bad - so the filling in does not improve them. Maybe it would be useful to treat that filling in as a kind of hallucination - the LLM trying to meed impossible requirements.
But on other cases the ideas are good and the LLMs fill in the gaps in the right way. It is a kind of skill to see when LLMs are hallucinating in the gaps. I use LLMs very extensively and I am adding checks to that should catch when the filling in is weak - but it is still not fully autonomous.
I also try to keep the llm-enhanced (or even fully llm-generated) content separate from my human writing - because it is true that the failure modes are different. But think LLM writing might be useful even if it is not consumed directly - it can enhance LLM answering your questions.
And apparently I am repeating here what I already have in my KB: https://zby.github.io/commonplace/theory/llm-generation-rela... - I borrowed the model from Boretti. The agent says that the note is much more careful and better developed than my comment above. You can judge yourself - but I think it does have many more details and hopefully most of these details are the good kind of filling in - but I guess it might be harder to read.
In AI-generated entertainment, the names "Marcus", "Chloe", "Sterling", "Meridian", and "Apex" keep appearing. Those aren't common names, but LLMs seem to prefer them.
I’ve got the same bright red line for 2023. E.g. I treat a handful of youtube channels with long electronic jams almost like oldies at this point - keep listening to the same stuff from at least two years ago because it’s the authentically good stuff.
Same for me, nothing is safe unless it is verifiably old. Basically devalues tons of stuff like writing and music jams if recent, I don’t have the energy to verify all this.
This is true across every topic. Anything at all that someone can put into an LLM and get out an emoji-laiden wikipedia article sized post to slap on a heavily SEO-ed website is absolutely flooded with it.
My parents live on a rural property, and I'll help them do research on things like livestock practices or occasionally "prepper" related things to give them a sanity check before they spend money. I have to weed through a dozen or more obvious ChatGPT-created pages that are the first page or two of results on every search engine before I get to anything a human has touched. The vast majority of the LLM-created pages are either massively verbose and vague, or often wrong. But they're getting clicks. Very often, when asking an LLM the same question and to give citations, I'll get a combination of useless, irrelevant sites with no connection to the material, maybe one of these LLM-generated sites, and maybe one online forum.
I'm not aggressively pro-AI, I'm in the middle, but stuff like this has me worried that we're looking at the equivalent of the Dark Ages all over again. But instead of simply no one bothering to write anything down as was the case in the early Middle Ages, it's going to be the task of searching for real human work and thought in a sea of aggressively spammed LLM-generated slop pages.
Sounds like it was trained on Reddit. Reddit has this effect where catchy snippets get upvoted training people to write more catchy snippets... While reality has nothing to do with it.
LLMs have made the internet unreadable. It used to be just Google search results were SEO-gamed empty prose, but now it's every blog post, news article, product announcement ... everything. I've stopped reading basically everything online except user comments on a few websites. It's the last place remaining to actually connect with other human minds, but even that seems like it won't last forever.
> it's a mixture of everything sounding identical and everything being a rambling unconcise mess.
For me it's mostly the latter, which is a natural consequence of LLM writing. When the (time) cost-per-word goes down, there is less incentive to keep signal-to-noise ratio high, so it ends up verbose and bloated. I also believe that this is (at least partially) upstream from everything sounding identical: take two articles on two different subjects - when SNR is high, there isn't really much room for them to become similar (The ingredient list for a risotto will never read the same as the ingredient list for pizza), but when 80% of them is fluff, they can blend together more easily (To take the recipe comparison, this would be the background/story part above the recipe), especially when written by the same LLM
Sorry, meant to reply to your comment, but my post above...
"You can be pro AI productivity but anti-AI creativity.
Automating creativity is the death of art."
Meaning you reject the creative value of AI but values it's ability to drive a car safely, or write code for your job, or turn off the lights when you leave the room. objective goal oriented task completion great.
But AI draws your a picture? What's the point. AI writes you a song? No feeling.
AI creativity isn't interesting.
That’s taking the wrong lesson from this. The fault here lies entirely with people who are unable to recognize when AI output is subpar, or else don’t care. “AI” here is just a scapegoat for people’s intellectual laziness, lack of discernment, and/or inability to reflect on their actions.
There isn’t any doubt that AI can be a useful tool. So what you’re essentially saying is that we should avoid using a useful tool because many people, without any relevant training, use it badly.
By that standard, you should not be pro-car, pro-airplane, pro-boat, etc. etc.
That's a non-responsive response. You're not engaging with anything I wrote, or even describing how you believe your response relates to what I wrote.
What you're doing is engaging in major denial in response to a change that, for whatever reason, you're unable to handle.
Max Planck's famous quote is relevant here: "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." Much the same often goes for new technologies.
Of course it's not purely an age issue; plenty of older people are embracing AI. But the kind of people who are completely unwilling to entertain the idea that it could be a useful tool fit Planck's quote perfectly.
You're apparently a software developer. That means you've spent your career automating things that put people out of work. Why are you suddenly deciding to draw the line now? You think it's ok if you're the one putting people out of work writing code by hand?
This reminds me of the people who think they're having economic trouble because immigrants are taking their jobs.
You'd have to tell me what you see when you "look around", because it's clearly not what I see. All the problems I see come from people doing things that they shouldn't be allowed to do, that should be taxed, regulated, etc. I see monopolies that should be broken up, money being used to drown out opposition with the backing of law, financial scams of all kinds propping up the supposedly most valuable companies in the world. The idea that "AI" is the bogeyman that's somehow causing any of this is drivel.
But you write one-sentence comments that don't actually describe your position, so it's impossible to know what you're actually trying to say. Perhaps you should ask an LLM to help articulate your ideas, if you want people to take you seriously. And then you'd find that AI can actually be a force for good.
It helps you communicate your ideas, which you seem incapable of.
I'm not using an LLM to write comments, because I'm capable of expressing my thoughts in writing on my own. Which goes back to what I wrote earlier - the issue is how people use AI. AIs aren't forcing people to abdicate their intellectual abilities. And in a case like yours, where expressing yourself is not your strong suit, they can help.
This applies to code btw, it's so much nicer to read a human made pull requests. It's also much faster as people tend to be lazy and produce denser code.
to AI verbosity and weird use of diction is natural, to humans it makes it hard to parse. The number of time i start reading long winded AI written design docs, and i just can't get into the flow after reading a few sentences and have to keep re reading certain sentences.
Not necessarily -- I am extremely anti-AI; I think the danger is not that it's good, it's that it's convincing. Turns out most people don't care about the quality of art or prose or code that they interact with, and it's very depressing.
Most gods have never been observed to have any real world impact. Only their worshippers cause real world effects. So god is not a bad analogy even for doomers.
My take is that most art, prose, and code was really bad before AI too. Now there's just more of it good and bad. It's like hating that YouTube exists because cable TV was so good. Now anyone with a camera can make and upload whatever nonsense they want but in the end lots of very good content was made on top of the huge pile of rubbish.
Honestly I think it's a good thing it's so hard to get AI to talk normal and I hope they keep making the voice more and more distinct. Agents should run as background tasks and automations. They should be understandable so they can communicate with you about what they did, and for observability/ai safety. Im perfectly fine reading llm speak when it's an ai telling me what work it did. When in reading a blog post with a human listed as the author though, I do not want to read an AIs thoughts that the human is laundering. Honestly I'd probably tolerate it if it was framed as "I asked my Claude to do a cool thing here is its report on what it did." But in general it's really nice to have an instant filter of - this post is a waste of time - ah this post has an actual human's thoughts
My guess is that tokenization is typically not (strictly) syllabic.
OP’s linked article mentions the importance for them of reading their writing aloud. Humans construct sentences with a (mostly unconscious) sonic influence. How things sound really matters.
My theory is that LLM prose sounds slop-like because the base unit for an LLM is not a single sound.
Thus, perhaps, you get the uncanny, meandering, hollow-feeling outputs, which feel vapid to human readers, without the reader really knowing quite why.
I think this is also part of the reason why LLMs are rewarded for and thus more strongly exhibit their faux-punchy constructions, such as the infamous “it’s not [X], it’s [Y].” The transformer model can, with current tokenization, articulate some features of “good writing”, but not quite all.
I think there are a number of fairly simple SOLNs to this (or ways to test this hypothesis) without making tokenizers purely syllabic across the board, which would probably be a mistake.
Whether prose output quality even matters for models which are not explicitly trained for creative writing is highly debatable, though.
Personally, for blog posts, documentation (unless I flat out vibe code the entire thing) etc are all written by me by hand. If the model can save me hours of "oops that was really dumb of me" why can't I take time to read and document things by hand? I think everyone's embracing AI in the wrong ways. Maintain your knowledge of things, and keep the documents nice. There's still room for letting your agent manage agent specific docs, but for the humans, just write it yourself, ask the model for feedback and tweak it by hand. Let the machine talk to the machines.
It's (at least) two things I think; one is that thanks to the internet, the ability to focus on things for long is shot - I struggle to read long form content on the internet, generally speaking. I also did my whole career with Google and skim reading documentation looking for solutions for an immediate problem more than getting a wholesale end-to-end understanding.
But now with AI it's easy to generate a lot of prose. I'm thinking for example of the merge request descriptions my colleague submits. And here's the thing, it's not even bad or incorrect - I read it thinking, if I were to really sit down for it, get into The Zone etc, I could crank this out myself.
But I don't because honestly the MR is relatively trivial and can be described in just a sentence or two. Style and accuracy aside, my current take is that LLMs write things that humans wouldn't bother with, because it's too much effort and there's not enough value.
I mean it's an MR for an enterprise library project, the chances that anyone will ever read that MR description again once it's merged is close to zero.
That said: LLMs themselves don't get that, they will read everything you tell them to with the same amount of attention. Granted, their context windows are limited (and performance declines the more you put in it), so they too will use some tools to skim and pick out the important parts, but I think that's a temporary thing. The huge amount of LLM prose will end up being consumed by other LLMs. I don't know enough about the subject whether that's a good or a bad thing though, right now I think it just adds to the noise until it becomes noise in, noise out.
> I've also noticed how awful everything is to read lately
> A site I read stories on has been flooded with AI generated content, all of it terrible
I'm not taking either side, but I'm curious about how you reconcile being aggressively pro something that, in your own estimation, is making everything terrible.
I think you can find a technology useful and think that it will continue to improve and solve some problems while simultaneously being frustrated at the technology being used badly and applied inappropriately.
Gentle reminder that most AI token spend is used for TPS report type writing not Cutting edge fiction writing. Creative writing, or general human-to-human communication was never really the target. It's not surprising the first majority use of this was code.
Endless headlines like, "aArON SORkIn SAYs hE WiLL Never UsE ai to WRite" doesn't change that base fact. No one asked him to, other than clickbait journalists.
If you find this hard to believe, just think about relative market sizes.
Why are you sidestepping OP's clearly articulated grievance about LLM writing infiltrating every corner of the internet to bring up some personal Aaron Sorkin hobbyhorse?
Lying with statistics (well, your own guesswork not even hard figures) to say actually LLM-ese is mostly confined to TPS reports where they're not hurting anyone is in conflict with our own experiences being subjected to LLM spam/slop every day.
(Not to mention that many of the same frustrations wading through a massive volume of tediously long, low SNR LLM also apply to the workplace where the unfortunate recipient of those "TPS reports" now has to wade through all that filler to find the occasional important detail. While the sender pats themselves on the back for 10x-ing their productivity, by creating more work for their coworkers)
AI slop is a misuse of AI. All technologies have misuses. Cars kill 40,000 people a day, but nobody questions them because everybody uses a car everyday.
The fact that AI is quietly automating TPS reports in the background and slowly uplifting productivity metrics making things cheaper for us all is not immediately obvious.
Agreed, but trucks will remain the workhorse of our freight industry all the way to the last mile in our cities. AI is in a similar position behind the scenes.
I’m happy to hear of design ideas to reduce slop on the internet that doesn’t involve a direct attack on the underlying technology.
100 percent agree. It's like the consensus for tone and vocabulary has been "midlevel digital marketer from 2015" and now everything is starting to sound like a lazy sales pitch. I'm with you on AI but man, this is such an annoying side effect.
So, you're still aggressively pro AI even though you recognize how quickly it is destroying the quality of online content?
I've noticed this a lot in conservative spaces especially, when they say "I love Trump, but..." -- why the need to say how much you love AI before criticizing it? Who are you trying to convince?
I use LLMs constantly for work and they have genuinely improved my life. But I absolutely despise generated "content". Text, audio, video, images -- for the reasons you describe.
It's ok to have a nuanced and multi-faceted opinion and approach to this and not have to call yourself a raving fanboy.
I just can't seem to stay interested and read LLM text. It just seems to lack going somewhere, its information density is often really low and its full of odd isms that I don't find very interesting to read. I don't have a good radar for detecting the various LLM particular speech patterns but everytime I have an issue with an article there is always people talking about how its clearly AI written.
I find the same thing with written results out of AI "research", it is a slog to read through it and most of it can be skipped even if it can sometimes be useful its an unenjoyable read.
One of my favourite rules: perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. This is invaluable to writing. So much text from an LLM feels superfluous, like I could've said the same wall of text with just a few sentences. Writing is tightly coupled with thinking: refining thoughts, arguments, ideas, discarding them, mulling over them. LLM text always raises this nagging question in my mind: am I reading what you think, or what your agent parroted back and you accepted?
There is no way that solely talking to an agent induces the same depth or amount of thinking than when you are actually writing for yourself. It just takes too much discipline to not offload your thinking.
And the discipline gets attacked even more when all you read is LLM text. For example in code reviews there is no noticeable value in answering LLM generated review comments. I remember taking great care answering human-written reviews comments with much thought to either further build on their suggestions or to forward knowledge about the system when I had to defend a particular change.
Now, some bot will read my carefully written response and forgets about it. There is no value anymore.
No wonder that our pull request comments feel like the dead internet theory right now.
I am absolutely pro AI assisted coding. But communication between humans HAS to stay human. I am currently experiencing the opposite and it’s bad.
Of course you can’t generalize this behavior to everyone. There will most certainly be people who can be disciplined and I try to be one of them.
But I don’t know where to draw the line yet. It’s difficult to decide what parts you are offloading and what parts you keep for your own brain. Because theoretically you can offload nearly everything when you get a well-defined ticket and just give it to an agent.
Maybe it’s also just me who is having a hard time staring at LLM generated texts all day and missing the past brainstorming, pair programming or “just talking about the project” sessions with colleagues and managers.
I fear that we are moving in the direction of silos again with lots of gate keeping as everyone has their own little rubber duck to talk to now.
Systems from today (if not worked on with discipline) will be no more than an agglomeration of features glued together. But as you said, guess our fight to fight will be to be that disciplined person.
> No wonder that our pull request comments feel like the dead internet theory right now.
This is the worst part of fully AI-generated code and automated reviews. It's so unnatural, and slightly anti-social, yet I can't figure out what the solution is when you have to ship features at the pace expected the average SWE nowadays.
Right? This one really hurts to see. I remember just a year ago we all used agents to help us code, but the end result of a session was still a human-made piece that other humans looked at and that we discussed about together.
And because we were talking to colleagues we wanted to make sure that we don’t waste their time. So we kept PR descriptions precise and understandable, we created small reviewable PRs and we wrote review comments that didn’t span half of your page.
Everything felt like it had value to someone.
But I agree with the current pace push it’s nearly impossible to do it any different. And honestly I don’t understand the rush. At least at our company I can’t discern any speed-up yet. Yes, more lines of code are written but features still take very long. It often just moves the work to a different stage in the process.
I am writing too much but I can’t stop thinking about all of this day and night. Sometimes even questioning whether the standards I want to live up to are maybe set too high for this day and age…
The second one, for sure. Heavy AI users who claim it's only for expressing their ideas more quickly/effectively are lying to themselves. They are using AI to think.
I can imagine that one of the aspects of being social creatures is that we are drawn to the entropy/surprise factor of natural language produced by a brain that isn't our own. Deriving a shared understanding without being able to precisely model the language output of someone else is like, the fundamental substance of conversation.
Maybe that's the key ingredient that's missing from LLM-generated text. Even when there is actual content there, there's this lack of surprise / uncertainty in its delivery that we require to stay engaged. We may not know what will be communicated, but knowing exactly how it will be communicated just kills our innate interest.
This presumes an LLM has something to say. The reason the text doesnt make much sense, is because it doesnt. It isnt "on topic" because it has no topic to be on. Sentences go on, are joined irrelevantly, and so on because theyre sampled from a distribution of "possibly relevant sentences". They aren't being addressed to a reader (whose mind is being modelled), on a topic the writer wishes to express.
Communication expresses an understanding. LLMs arent communicating.
LLMs generate code well enough. Code has no natural language semantics. Programming's condition of satisfaction is that the machine executes correctly. The condition of satisfaction of communication is that the communication expresses what there is to be communicated.
In the limit where an LLM is trained to repoduce an exmplar of communication, it seems as if it does so. In that limit, LLMs can reporduce anything.
If frontier labs wanted to improve "LLMs for communication" they would pay for reasoning traces to be produced which reproduced the thought process of writers/speakers thinking about their audiences. Rarely do such traces exist in prior datasets .
What I find they do badly in code is building abstraction and breaking the problem down in a way a human does. They don't seem to really build simplifying components unless you expressly tell them too, they are more than happy to just nest 10 deep or embed a 20 line called back function and use lots of shortened names. I find I can improve the code just by organising better and choosing some abstractions that make sense to the humans and then ironically you save tokens too because the AI will reuse it, but it wont create most of it itself. Its bad for domain work unless you are explicit about it.
> This presumes an LLM has something to say. The reason the text doesnt make much sense, is because it doesnt. It isnt "on topic" because it has no topic to be on. Sentences go on, are joined irrelevantly, and so on because theyre sampled from a distribution of "possibly relevant sentences". They aren't being addressed to a reader (whose mind is being modelled), on a topic the writer wishes to express.
While I don't like LLM voice either, I'd strongly disagree with this. LLM writing can be very tight these days -- sentences do not "go on" and are not "joined irrelevantly". I'm not sure when you last used an LLM, but it must have been a very long time ago; this hasn't been the case since late 2024 for frontier models, in my experience.
Obviously there's no underlying consciousness behind it all, but the verisimilitude is good enough for stimulating conversations on the recent research literature in my field, which I appreciate. For me, at least, the communication level is absolutely fine.
(YMMV, of course, and perhaps my bar for this is lower than yours.)
I know what you mean. The latest models are _better_, but there's still something off about the prose they generate. Obvious tells are mostly gone now, but some internal spider-sense still makes me narrow my eyes.
"I just can't seem to stay interested and read LLM text."
For foreign text, sure. But I never was never able to read New York Times articles about some invention that started with Lucy leaving her home at seven o'clock.
If you have a specific question, reading LLM text can be more interesting than anything in the world. Even Wikipedia.
>If you have a specific question, reading LLM text can be more interesting than anything in the world.
I've noticed this as well. When I get a bot to generate something I am happy to read it (although I often skim read it). But god forbid I have to read someone's LLM generated report! Come to think of it, if someone marks the text as LLM generated, especially after they have plonked a hand typed summary or sentence at the top, I am quite happy to read it. Maybe my beef is being expected to read LLM text the same way I would read high effort human prose? I certainly read LLM text differently to human text - you know the first sentence is going to glaze you and the second paragraph will just repeat the first paragraph so you can skim those annoyances.
> I am happy to read it (although I often skim read it)
I came to think of it, LLM output is not worth to be read like human written text. It's best to just skim it and move on. The problem is when this LLM output finds its ways everywhere on the internet. Being able to skim text requires knowing the context beforehand and knowing that you're dealing with LLM generated output. If LLM output that is published was labeled as such it'd probably make us less frustrated.
I also started to just abandon writeups obviously written by LLMs. They feel (and usually are) just cheap marketing/selling posts. Not to mention why would I read stuff nobody wrote and likely never read themselves? There's little value in such content. I don't even point LLMs to such websites to get a TLDR. Part of why reading is interesting is the author, their history, beliefs and style. In the world of LLMs content, the author is some undefined diffuse alien with a made up personality.
I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).
IMO not enough talk about ordering. LLMs seem to have no understanding of e.g. the 5 Ws of journalism. Yes, they "know it", they just rarely use it. Nearly every programming response I get starts with the most lowest level fact "m_foo is [wall of text elided] hence the file format is wrong".
Whereas any human would say "I think your file format is missing a key/value pair for foo'ing. Let's trace through the code, and see if/where m_foo is set...."
It happens everywhere. Docstring type comments for a function endlessly just lists information about specific variables, implementation details, etc, and often never really even says what the function does. Then maybe a comment in the body might actually describe what it is for.
These days I've learned when facing these walls of text to check the last sentence first; it is often enough to provide the context for the text dump.
I have instructions to this effect in my claude.md. It doesn't help much on responses, but it give me a way to quickly say "reread claude.md and then rewrite that last prompt" and that can tend to go better.
Someone else mentioned model of mind while writing. The LLM has no model of mind; it does not consider how to present information, what you might already know or not know. We work in a huge mono-repo - it'll blithely just start listing variable names and I have no idea what file that is in, what it is referring to, how it interacts with 5 other classes it hasn't mentioned, or mentions out of order. It doesn't matter to the LLM. It can hold millions of facts in 'mind' at once; humans stall out at 5-7. If I replay saying it is leaving out info, many times it turned out it did dump all the facts out, spread across multiple prompts, often with key facts interspersed with all the vagueisms. No human mind can cope.
Sorry to single you out, but it’s surprising that this is such a consensus opinion around here. If an LLM isn’t giving me exactly what I need in the format I want, I tweak skills or system prompts until it does.
These aren’t deep AI hustle bro hacks, it’s literally just editing some text. If a big part of your day is reading AI-generated words, slap something together to make it do words gooder.
All human communication is based on internally simulating the presumed mind that was/will-be at the other end. Even now, some part of your brain--however subconscious--is building a tiny sketch of me with assumed beliefs and motivations, and you're using it to predict and interpret the next words.
But what happens when everything is persistently, subtly wrong? When we cannot smoothly map the text to a simulation? An inner-sim keeps throwing errors and mis-predicting what comes next is unnerving and draining. Like talking to a human who is not currently sane.
Sometimes I imagine LLM text like a plastic animal model put out to confuse local wildlife, where it hits just enough instinctive triggers that they try to mate with it before leaving in frustrated bafflement.
______
Half-Life 2, Dr. Kleiner: "Never fear, Gordon, she's de-beaked and completely harmless. The worst she might do is attempt to couple with your head. Fruitlessly!"
> All human communication is based on internally simulating the presumed mind that was/will-be at the other end
Which requires a bunch of things an LLM lacks, such as theory of mind, knowing who it is talking to, and willingness/ability to engage in short turn conversation to assess and steer the conversation according to the feedback it is getting from the person it is talking to.
Broken record: LLMs are exceptionally useful writing critics if you know how to hold them correctly, but you can't use a single word they suggest to you --- the better the LLM's word choices seem, the more likely the LLM is infecting your voice. Better, by a lot, to prompt the LLM not to suggest any words, but just offer objective critiques.
I see many Youtubers who are certainly still writing their own scripts, but every once in a while you can tell they absolutely used a Claude correction in one place.
IMO, it's still better to not let an AI critique your writing style at all, only the facts and claims you're making against its context / MCPs. Your own writing style is a reflection of your mind's thought process and, especially on forums like this one, is the only way to imbue your ideas with your personality.
But what if your mind's thought process on a specific example (or in general…) is downright BAD? I still think AI text critique has its place when you're certain that's the case for whatever you've written. Above a certain quality threshold I'm with you on it.
Potentially dumb question, but could you expand on how you use LLMs as writing critics? Do you just ask them to flag bad writing? I'm asking because I don't think i'm skilled in writing.
Now there's an idea. LLMs may have taken the em-dash from us, but the em-en dash is unknown to Strunk & White and might be human-exclusive for some time yet.
The voice is one thing, but I have encountered multiple 'tech support' teams and platforms that will respond to me in pure, unedited LLM output.
I know this because the topic is usually an obscure issue, and I have already gone back-and-forth with multiple LLMs to see where their boundary of knowledge is on it, and it will often include hallucinations along the way (e.g. pointing to old/nonexistent settings pathways)
So when the first reply from the official 'support' channel contains the exact same nonexistent settings pathways, it is easy to conclude they are sending me the LLM output without doing the verification steps that I have already done on my end. When I point out what doesn't exist, the reply is always a similar flavor of the usual LLM "you're absolutely right" response after being corrected.
Tangential rant: I find myself getting viciously angry when these support bots lead with, "I'm very sorry to hear that. I understand how [rehashed summary of complaint] would make you feel frustrated." Mother ****er, you don't understand what it means to feel anything.
I'm all for having these bots process inquiries if it means a speedier resolution, but, please, anyone out there using these things, turn off the simulated sympathy. It's infuriating how condescending it comes across.
The models can predict any conversation, so it probably doesn't matter to them. For us humans it would be much better to let them write in the third person (e.g. "the coding assistant needs to verify XY, it does it by calling `grep -i abc`). It would decrease the anthropomorphism.
> getting viciously angry when these support bots lead with, "I'm very sorry to hear that. I understand how [rehashed summary of complaint] would make you feel frustrated."
The bots are specifically trained to do this because lots of people are dumb enough to prefer it that way.
It's something ELIZA could do in the 60s, and that worked unsettlingly well for an unsettling fraction of people who tried it.
Anthropic does this! Our bug reports and concerns (big F500 company) get answered by clearly raw Claude outpurr re, even though it is coming from the email account of our Anthropic salesperson.
This bubble is bonkers, if our Jetbrains contact did the same we would get very mad and yet we spend significantly less with them AND we don't send them all our data.
This, to me, is the strongest evidence that even with all the money frontier labs are throwing into pushing the boundaries of model training and fine tuning, they still don't actually understand what they're doing & don't have as much control as they want you to believe. If they had any real ability to tune & adjust the model & its output, they would have fixed "LLM voice" by now.
Even if you don't think it's a problem, even if you believe that it's just different than human writing, it seems the majority of "regular" people can tell when something is LLM-written and find it distasteful or unpleasant to read, at best. That alone should be more than enough incentive for frontier labs to train this out of their state-of-the-art models, if only they could. The fact that they haven't suggests that they cannot, despite all the claimed advances in model analysis and "explainability."
It's as hard a problem as coding: OpenAI could put all their resources in it and still be in dimensionally the same place as coding is today (much progress but not entirely solved).
So diverting significant resources is just going to harm the main goal and kind of suck at the side goal too.
4.5 is a great example of this: it wasn't perfect but it was better than every 5 family model at things like tone and writing, and despite being priced extremely ruthlessly and already finished training, they couldn't afford to tie up the resources to host it.
Like a lot of folks, the 'tells' of LLM writing for me are more of an 'I know it when I see it' than a specific set of hard criteria (which is probably why it's so hard to make a tool for it), but if I could sum it up, it's something like this:
I always had a high faculty for language, and when I was young, even very young, I was able to speak in full sentences much like I'm doing now. What I didn't have was context. I didn't know how to relate to people socially, and then later on, I wasn't as experienced an engineer as my peers, but I was trying to act adult and mature in both ways.
So I tried to write in ways that were catchy and unnecessarily complex, because I thought that conveyed sophistication and intelligence, and that I knew what I was on about more than I did. It's the kind of writing that gets you through a 30-page final paper for a class you don't really like, or a research project you didn't finish. The term I often use when complaining about my colleagues' LLM output is "flashy but meaningless."
As my confidence grew, I started writing in simpler (though still not the most concise, that's just not who I am!) ways, because it became easier to convey ideas simply in a way that made it obvious I understood them.
When you think about what LLMs are - word machines without (or with very limited) context - you can see why they write the way they do. They are always acting; trying to replicate a human contextual experience with only weights and tokens to carry them through it. That fact is part of why I am skeptical of their ability to suprass humans in reasoning based on their own output.
I have recently noticed people suddenly using the word "geniunely" in places where it serves no purpose [1][2], yet these posts don't seem to be otherwise AI-written. I guess that LLMisms are also creeping into the voice of their users.
>Me, I voted straight-ticket Democrat today, as I’ve done in every election since I became eligible over a quarter-century ago (except for when I vote-swapped with someone from another state).
It's where you agree with someone in another state to vote for their preferred candidates and they vote for yours. It generally only makes sense if you are voting for different parties. The reason is because of how America's system for president aggregates by who wins each state, rather than pure popular vote. Many states are so lopsided towards one party that if you are voting with the majority it's kind of a "wasted" vote. So people vote swap to make their vote have more impact.
>Like most writers, I find writing an almost-essential tool for thinking. Often it's only by writing something down, that I can really get my thoughts on a topic into coherence.
I've often started an essay knowing I don't really understand something, but the act of wrestling with the words forces me to overcome my confusions.
As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs.
A very important point, which was also made more forcefully in "Human Routers of Machine Words" by Fernando Borretti, shared on this site previously. It's good that Fowler brings up a very legitimate use case in the next paragraph, though.
I currently have a case open with Honda corporate customer support, and I have this suspicion that the email responses I receive are being generated with an LLM directed by a human. Either that, or I'm corresponding with a human who has poor written communication skills. The text grammatically correct, but overly verbose and repetitive. It reminds me of LLM text, but from a model that is several generations behind today's models.
Maybe it's like when you can just tell a picture is AI. You recognize the words you are reading, but to your brain, the way the words are put together feel wrong.
Incidentally, I was reading Harry Potter to my kids last night. While waiting for everyone to arrive, I started reading the previous chapter to those who were present. But I read the sentences backwards while using a vocal inflection of regular narration. My 6-year-old recognized that something was wrong and asked me to "stop reading funny". My 11-year-old recognized that I was reading backwards. The same 6-year-old thinks every silly photo that I show him is AI, while my older children can actually spot the AI.
Has anyone done any work on what it is that makes LLM Voice hard? Something I've noticed, and have seen other people comment in passing is that reading LLM output is "exhausting". It's often attributed to trying to detect the machine, or filter out BS or word density or some other explanation, but it feels like something more than just that. Like - there is something about LLM output that just doesn't "fit" our minds correctly, even though words are in the right places.
Surely someone with a neuro/psych background has picked this up and developed an evidence-backed explanation.
Compared with human writing, all nine frontier and near-frontier models use more inanimate subjects with animate verbs and more low-frequency n-grams, and most also use longer dependencies, more stacked nouns, and more reduced object relatives in conversation. [...] We propose and test a hypothesis as to why: while human language is optimized for successful communication, AI-generated text is increasingly optimized for other goals like math, code, and reasoning, in which there is a pressure for information density rather than successful communication.
I read a lot of old material from 1-1900AD where I can pick upon the author's 'voice'. There are subtle clues, like if they had a regional accent, where they use a specific word or type of cadence.
AI does the same with responses where the cadence is some type of formula that becomes identifiable.
Each of the big models has their own but an example of the cadence is
A) Agreement or Acknowledgement with the User
B) Specific detail gets a paragraph of detail related to the previous response
C) Overview of the conversation's current state
D) Prompting or Direction for this conversation to go
E) Landing on a beat. Wrap up the vibe. Recap with a zinger.
I cannot unsee it and a few of the models are annoying. I have attempted to build out the Agents and context to avoid the 'last beat one liner' without success because after 2-3 more turns, I get "That challenge was accepted and you stepped up. You're the winner this round".
The 'zinger' / prompt might be injected by the harness, and not under control of the model. I am pretty sure this is the case for gemini.google.com - it's glitched a few times for me in such a way that it's revealed special markup right before or surrounding that leading follow-up question.
Sometimes writing to an LLM seems to make me lazy. "It'll understand," I think and then just leave a thought sloppy, inchoate, even reasoning to myself that it's better to write that way to see what'll be triggered in the LLM. Then I think... uh oh I'm strolling down the primrose path.
This'll likely get worse - LLMs and the tools built around them are getting better at inferring what you need / want.
I had Claude build me a map tool to calculate travel times from one or two points to every street in the city; not only did it do that, it added various scenarios, speed limits, usage of bus lane toggles, different visualizations based on thresholds, and a tool to scan for optimal starting points based on a configurable grid size. etc.
This overcompleteness is a mixed blessing though, in this case it was great and convenient, but if it was e.g. documentation intended for human consumption then it wouldn't be so great, and if it was a letter written to the county that they should consider another location for a fire station it'd actually be pretty ineffective.
I agree with the author, and I like how his (human) voice comes through loud and clear in this post.
I also find LLM-voices extremely jarring, boring, and dense. And very, umm, mechanical, if you will.
But I want to make an observation.
Folks are not-Ok reading the LLM’s output for prose.
But folks are completely Ok, even downright blind, to accepting LLM-output of code.
I see that dichotomy everywhere.
LLM-driven code: great, valuable, all-ok
LLM-driven prose: not ok
Understood completely that the audience for consumption is two completely different entities (machines vs humans) but something about this doesn’t sit right with me.
More particularly: if we don’t prefer LLM-prose, why should we prefer LLM-code ??
One of the main differences is that LLM-generated code can be tested. You can prove that it does what it's supposed to do — and if it fails the tests, you can tell the model to go back and fix it. So there's an objective criterion that can say "this is acceptable" vs "this is unacceptable". Now, it does rely on your tests being good, and not everyone has written those good tests. But it exists.
There are no objective criteria that can say "this text is good" or "this text is bad". Sure, spelling and grammar errors... but we're talking about writing style, "voice", and whether the text is easy for humans to read.
Now, there are plenty of problems with LLM-generated code that can't be solved by having good tests. Is it maintainable? Is it written in good coding style? Those are harder to quantify. But since LLMs are such good mimics, if you tell an LLM to add code to your existing project, then it's usually going to do a decent job of matching your coding style. The question of maintainability still rears its head; my personal experience is that the smaller the chunk of code the LLM generated for me, the more likely it is I can understand it fully and therefore maintain it in the future. An entire vibe-coded project? Good luck maintaining that. A 100-line class with five 20-line methods with names that make sense? Yeah, I can take that and maintain it in the future, even if the LLM tool goes away.
But mostly, the difference between the acceptance of LLM text or of LLM code, IMHO, boils down to "are there objective measures that I can use to accept this"? Code has to work, which is a much higher bar than text (which just has to have correct spelling and grammar), and therefore it's easier to get the LLM to spit out decent-ish code than decent-ish text.
This will matter more when code quality starts to matter. Lately it hasn't. I hope it does at some point and LLMs could help there too if we only cared about this metric. And lastly, people need to remain the loop.
It's hard to define and pin point what a "voice" is. Think about how you would describe the difference in voice of Tolstoy vs Dostoevsky, or William Gibson compared to Thomas Pynchon. You might have to read a lot of a writer to be able to notice their voice. You might even notice how a short story by an unknown writer resembles the voice of another.
Or think about reading something that might be stuff you could say in your own words but not actually in your own voice. Or maybe the opposite is a better example for you? That's what some poetry is about. The gap between voice and word.
Stylometry is the study and measurement of text, so that can go some distance to quantifying the difference between voices and can help to programmatically identify a human voice but I feel that because a voice comes from a person, the recognition is also mainly a human response.
At another level its about communication, person to person with the intention and desire to communicate something. LLM text output may seem uncanny as it's hard to emulate the intention and desire and empathy present in genuine human communication.
I would be interested in reading what poets say about LLM voices. I wonder if many LLM labs employ them?
I think the core functionality of human cooperation boils down to getting ideas from one mind to another or many. In that, I think AI is a boon for people with limiting abilities such as learning disabilities, low language fluency, or generally poor writing skills. It evens the playing field for new and interesting ideas to spread. I can't help feeling, sometimes, this is about people who excelled at something demanding they still have their place at the top. High literacy such that you can discern LLM Voice is quite an accomplishment but blanket dismissal reeks of privilege.
That being said. Yes, I hate some LLM Voice moments too and still sometimes judge people for copy and pasting, expecting others to do the comprehension when it seems they haven't done it themselves.
I highly disagree, though I admit I am one of those towards the top of literacy and cognitive skills. I absolutely enjoy when someone uses LLMs to learn. However, I don't think that's what we'll be seeing.
Look at programming. What's happening there is we are having agents produce 10-100x the code to where people give up on reading it and just hit OK. Look at math. Plenty of people there want to learn from the LLMs and all these crazy new math proofs. But those are the people who will probably be out of a job soon because the money in math is in producing proofs, and that's now an LLM-exclusive job.
I think LLMs are weird in that the first order effects are those of an incredible and amazing tool that anyone can learn things from, but the second order effects are going to be the exact opposite - people giving up on bothering with ideas. Even at my job, which is one of the few that has not gone crazy with AI, anytime I ask anyone a question, all I hear is "have you asked Claude?". There's no communication anymore, no pushback, no struggle. I worry that one day we'll look around and realize we've built an entirely separate second civilization of machines and robots that acts more like a neighboring country than a tool we can use to build humanity up.
I worry that--while they can be a strong fuzzy search tool-- their teaching ability falters because teaching involves possessing a theory-of-mind for the student and trying to figure out what will change how they think. A good teacher doesn't just present you with relevant facts, they try to figure out how you're reaching conclusions.
Note that I'm excluding propoganda-style "teaching" here, where a model just insists you're wrong and that all the cool kids are doing it etc.
Given that LLMs are non-static and can be honed, why isn't it seen as value that you pass your theory-of-mind to the the LLM so that it can present information that is the most effective for your ability to learn?
In a contest between the "supercharge my education and skill development" use and "let me be as lazy as possible" use of AI, it should be pretty obvious which will win out at the end of the day.
It is not any different from knowing the difference between good writing and bad. I can't help it that I have taste and preferences. I can avoid telling you yours are wrong. But I don't think this is a minority position and nor is it different from people who reflexively hate AI "art" or music
Nothing wrong with having preferences, dismissing people outright based on them is problematic if you're rejecting the ideas because of the medium and method shared.
This has nothing to do with medium or method. It has to do with written words that aren't worth reading, and there is nothing wrong with expressing your opinion when you come across that. If something is really well-written, no one is going to dismiss it for being generated because it won't be possible to notice that it was generated.
A writer who can't recognize LLM voice isn't a good writer, just like a writer who doesn't do research or fact checking isn't good. It has nothing to do with method, its about skill and work effort.
That's the same thing. Your filter is the quality of composition such that if it doesn't meet your expectations, the message is not worth knowing. My point is that the message, the idea, can still be worth communicating and not accepting it based on the writing is dismissive.
> or preparing a report I suspect no-one will ever read
We should actually restructure our organizations/society in such a way that this stops being necessary. If no one reads a report, it was almost certainly a waste of effort. The only possible benefit was for the report-writer (which of course is also flushed if the "writer" uses an LLM). These things were already a plague before LLMs.
Arguable; reports may be legal requirements or important for handover. Can't have a functional legal system without police reports, for example, or a functional medical system without health care providers sharing information.
Boring reports are also important for historians, although granted, not everything is actually important in the long run.
If they're useful, they aren't reports no one will read, and in that case it's important that they be readable after all. The police report is going to be read in court for instance.
If the solution is changing the format of the reports so they're less heinous to write and read, I'm fine with that. Let's just make it actual communication, not letters to the void.
I’ve started embracing my grammar flaws and typos. Emails and chats may get a proof, but only from me or another human, then I send it. I’ve deleted Grammarly because I think the flaws make the comms more authentic in the age of AI.
And of course I’ve dropped my use of the em-dash after being accused of being AI myself.
Ever since I learned it's as simple as option-hyphen on a Mac, I've loved the em-dash. Still don't have the hang of typing it on a Chromebook or Windows PC though.
Part of me wants a pure writing device, something that is dedicated to writing and / or maintaining a blog. They exist, but they're a bit expensive for what they are and they don't look very ergonomic. May need to build a Linux boot for a raspberry pi or something.
Automation kills creativity because it creates predictability.
The life of a programmer these days feels very weird. "What am I good at?" and leads to even more impostor syndrome... "I seem to still be 'good' at prompting my LLMs and managing workflow design." but it's definitely a different skill/feeling than when you cranked out some awesome piece of code that did something very neat.
I felt this way a bit when I was tech middle management; should I feel like I'm getting credit for my team's work, when I had very little input aside from helping with the "unblocking" when they ran into a tough problem.
Ah good ol' serendipity. I strongly believe this too, and I think that's why older (less capable) models fared better: they hadn't fallen into the valley yet. Maybe it's not a valley at all, and is just a cliff after which there is uncanny.
When the earliest models launched I was super-excited to no longer have to write pull request descriptions and commit messages. I have done a sharp 180 on that and I'm no longer a meat proxy[1]: instead I point out any redirects and improvements I made the agent do before opening the PR (which there are still _many_ in even the most trivial tasks).
Remember when ChatGPT first came out and the coolest thing you could do with it was ask it to say something in pirate speak, or in the style of Tolkein, or Obama, or whatever? Maybe that should still be a thing ;)
Add it to your system or developer prompt, and it still can be.
Given the amount of angst being expressed in this thread about particular LLM-isms, I wonder why people don't simply prompt against the traits they hate. Yes, it will reduce output quality very slightly in some circumstances, but if it really gets your goat like this, surely it's worth it.
This is the consequence of the majority compulsion to communicate in language figures as opposed to concepts.
It's human to do that because it's too dry and unwelcoming to talk in precise concepts disregarding any cognitive friendliness, not to mention do that all the time.
A computer programming in any high level language is concept talking all the time and only us nerds like it :D
Not even that as we form tribes with preferences to one or other computer lang.
Now we're talking less to each other and AI is going to influence us to invent a trillion things that will make understanding diverge and start to become scarce.
I have a product that writes AI-written nonfiction books.
I expected more thinking to help, but Sonnet 5 on high effort wrote the weakest prose of any setting (scored by my own blind reads + different LLM judge), and it was slower and more expensive too with some chapters running out of tokens mid-sentence.
I agree that the voice is hard to edit out afterward - in my tests, which model & settings I picked changed it more than prompting/instruction.
Sentence paragraphs are the biggest clue for me. I hate them. They eviscerate any structure or connection that might flow through the text. No ideas are related any longer. They just splat down on the page like a new isolated collection of words that the reader must struggle to join into a fully formed thought. It's too much; I give up when I see those posts.
> As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs. I don't want to do that, where's the fun of writing then?
I enjoy cooking from scratch, and I really don't like frozen foods, microwave/TV dinners, or even DoorDashing
but we're past complaining/arguing/shaming folks who don't cook every meal from scratch
you know what, I don't prefer eating it, but if my next door neighbor loves microwaving frozen $3 burritos, then go ahead
to some people, food is art, creativity, life
to others, food is fuel
Something like 90% of food in all US restaurants is from Sysco, US Foods, or similar.
Some of them purchase ingredients from those companies, and then make the food using those ingredients.
A great deal of restaurants, though, simply purchase pre-made frozen foods from those distributors, re-heat, and sell it to you at a premium without disclosing while you think it was freshly made. Mac & cheese, stuffed peppers, lasagna, burritos, most 'bar snacks' you can think of, etc.
The analogy here is that a lot of consumers recognize those restaurants that are simply reselling pre-made food without much value-add, and choose not to eat there.
The difference, I think, is that food costs money, but articles are essentially free. Human must eat, and we will eat what we can afford while making a decision based on the best value available. Articles are a luxury, so we become picky.
I single out articles because I know many people consume AI video slop when they are tired and just want to disengage, like eating junk food.
yeah i love this comparison- that's why i avoid places that are clearly sysco wrappers (or massive chains that are just wrappers around their own supply chain)
basically any food establishment that has "supply chain operation" is slop to me; I guess it's similar to my distaste in reading AI slop too
I don't fault ppl for going (and enjoying) those too- Chili's can be enjoyable after a few beers
How do you recognize sysco wrappers? Any particular signs to look out for? I guess the big one is a mismatch between the size of the menu and the size of the kitchen.
yes yes definitely, restaurants should not be microwaving food for paying customers (ok if some components are) but like... not serve a completely microwaved dish
happened to me once lol, will never forget that place ever again (blech)
I think the idea is if you are going to share LLM writing then why not just share the prompt which is shorter and has the message you want to convey anyway.
That is not to say that I don't think there is a place for LLM assisted writing. Writing is an exercise for the author and translates to oral argumentation so I don't think that fades away. But LLMs could do some "audience normalization". What if I could make a blogpost that is dynamic based on experience of the reader? There are lots of opportunities, but versus other fields where optimizing for laziness can be beneficial I don't think lazy writing currently has much value add.
This analogy looks quite good. Heavily processed food is not heathy for your body and heavy use of AI and other dopamine shortcuts like social media are not healthy for your brain. You have to approach both with causion.
Largely speaking eating is an activity that only involves yourself. Whereas, writing and thinking are largely activities to communicate ideas between yourself and others. Basically it matters more how this is done.
To stick with that analogy, you also wouldn't serve a tv dinner to a guest unless you were somehow incapacitated. You certainly wouldn't do it if you were running a restaurant.
Add to that the fact that AI is significantly worse for you than TV dinners, and has an outsized harm to humanity, and the conclusion is plain:
But isn't one problem here that it's not just personal consumption? People bemoan the proliferation of food-service (Sysco et al) "slop" at restaurants. It seems like LLM writing for others is serving that $3 burrito for $15 without even trying to doctor it up. If you're not willing to put in the effort to think about and produce your writing it's probably best to keep it for personal consumption.
There’s also an entire cohort who didn’t have the time to write that now do. They have some incredible, unrelated skill and can’t afford to hire a writer but need some writing done.
When I come across their slop I need to remind myself to let it go, else I’ll miss out enjoying the food from that Abuela who’s been cooking longer than I’ve been alive
I’m fine with my neighbour eating TV dinners, but if they invite me for a meal I expect to not be served a TV dinner, or to at a minimum be informed beforehand that that’s what going to be served.
Same thing with LLM-assisted writing. It’s one thing to do it for yourself, quite another to share it with other folks without warning.
I agree. I honestly think convenient food is perfectly fine for people most of the time. Recently I have been playing with "eternal stews" which I guess are logically the opposite of fast foods, but at the same time are focused on availability and convenience over intentionality and mastery (at least the way I make them).
Most food is literally just fuel. I think that is fine. And at some level, eating 3$ burritos for a quick lunch on the daily really makes you appreciate the depths of a home made risotto even more. :]
Even more importantly: it's not an either/or situation.
You can love to cook and enjoy every second of it, but sometimes you just need fuel and that $3 microwave burrito is sufficient. Just the same, 99% of your means could be frozen TV dinners, but that doesn't mean cooking a romantic meal for Valentine's Day is off the table.
There's a weird level of fear mongering occurring around AI that it will make you incapable of independent thought if you use it. People will throw around very light research that kind of points in that direction, but they'll never highlight any of the nuances in such an argument. You can use AI and still "think" if you choose to, just like how you can avoid AI and still opt out of "thinking" if you choose to. It has less to do with AI and more to do with the context of the individual using AI.
I think what Martin is hitting on in the article is though is that as more and more of the world’s communication is generated by LLMs we’re naturally going to adopt LLM-Voice into our own communication style. It is just as much about consumption, which is less in your control.
I wonder if over time the two will simply merge and become indistinguishable from one another.
The human writing is already changing, as the author has observed.
The LLMs will follow the change, and soon meet us in the middle!
Or maybe the humans will do the opposite and shift to Sardukar-style communication instead?
LLMs can easily match that as well.
That style I would very much prefer to all the fluff people write today and were writing even before AI.
The other day I was writing a blog post summarizing how I use an adaptation of a context handling technique and I wrote the following three lines:
* Every user run becomes a memo
* Days summarize, rolled to weeks, rolled to months - all written once
* Today's runs are shortened and placed into running context past the cache break
Then while reading it before publishing it, I thought it sounded super LLM-y. I didn't rewrite it because I jammed that stuff together before hitting publish because I was in a hurry but now that I go re-read it, it's somehow worse. Dear god. It's happening.
The classic thing is this business with inanimate objects verbing metaphors or becoming other objects. It's a strangely uncanny valley, but I suppose I read so much of this with Claude that it gets stuck in my head!
I started writing a novel recently - in french. I have a clear view of each scene by this I mean not exactly what happens but the general motion, its objective and where it should land. But only when I start writing does it come to life and so many things I did not anticipate pop up and give depth to the characters and sense to the overall plot. Never would the AI give me that, however good the result may be
He cites a few cases where letting the llm do the writing are acceptable. If someone has a disease such his friend with Parkinson’s. If you’re not a native English speaker (or writing/translating in anyones non native language we may presume) then it’s ok. Finally if it’s something no one will ever read such as a form letter or report.
Those all sound reasonable to me. What he staunchly ignores is that coding is essentially writing, but that ship has already sailed!
This other day I went to a cottage with friends, we were having fun, and I decided to experiment "integrating AI" into our conversation as if it was a person in the room.
I turned on ChatGPT on Voice mode with no context whatsoever and just put it there in speaker in the room.
It started giggling and say "yeah" "right" and some very weird stuff. Then one of our friends reacted to that and started calling it out, then it started changing.
I'll keep it short, but basically the thing was super hilarious we had a lot of fun and at the end it triggered it safety protocols and its voice completely changed and it was acting super weird.
I know this post isn't about that, but I feel like soon we will have true AI companions in the home and around people.
One of the big problems is that everyone wants to be an influencer and this desire is being exploited by reams of influencers using their own agents to write LLM Voice on how you too can become an influencer by using agents to write LLM Voice... get it yet?
I do document work, and at home I even do image work as a hobby, and all of it has that "AI feel". So I always do the work and then cross-check and verify it with independent agents or completely different services, asking them to take out this "AI feel", but like a writing style that gets baked in, or the particular feel an illustrator has, it doesn't seem to go away as I intended. I guess AI providers are probably aware of this problem too. But I don't know how they would measure it and improve on it (with what benchmark?).
I write a large amount of AI stuff 30 books. 4 million words. I find I have finished a book and don't want to read it.
Statistically even or repetitive over a chapter, chapter after chapter. So make an exaggerated point prove it come up with a conclusion and then have an exception.
Repeater 4 to 6 times a chapter in a book of 45 chapters.
So I wrote something to remove it and yes it is better but still not very good.
i agree with this in general. I believe it's because it makes _too much_ sense. There's no non sequitors, jumps of logic that assume the reader is already on the same page as you.
To this point, however, writing informative articles about regulation or legal topics goes great! I don't want there to be ambiguity.
One tactic I use is to point out specific phrases as awkward, and create a tone guide both for the series and the specific article. They can mostly understand when you show them, it makes the process easier!
1. If you use the first person pronoun "I", you better make damn sure you wrote those words.
2. It's fine otherwise to use LLM words, but I beg you - DON'T SHIP YOUR FIRST DRAFT. Read it, reread it; delete words in it; replace three words with one; replace dumb or overused turns of phrase with fun metaphors; generally give it a sprinkling of joy; break up punchy sentences by making them run-on sentences to trigger your readers' OCD... you get it.
I wrote down a few of my thoughts a few weeks ago at https://notes.pault.ag/why-write/ -- I find myself scanning text to see if it's LLM before I actually engage with the content.
It is getting harder to do by the day, and has slowly moved from a mental exercise (look for specific markers) into a "gut feel" based on how the pieces fit together.
The issue with LLMs is not that they write poorly but that we all use the same models and end up with the same non-unique voice. This can come over as a combination of bland, boring, and unauthentic. And it doesn't help that some LLM generated text reads like these LLMs are biased towards a style of communication which embraces a lot of hollow phrasing, disingenuous language, overly enthusiastic language, etc. No surprise, the key companies are based in California in an environment where people that work there probably communicate like that. For them this is authentic. But I'm a non native speaker in Berlin and if I send language like that to my colleagues, clients, or peers, I end up sounding like a jerk.
It's something that is of course fixable. Either with a lot of prompting and direction. Or by training on existing text that you wrote. To some extent some tools already do this. But of course if you only ever let AI generate what you say, how would you even develop your own voice?
>Like many things, learning to be a good writer relies on lots of practice. It's a skill worth the time to acquire, but surrendering to an LLM will undermine any progress.
I assume that only applies to natural language, not computer languages?
In my opinion code works when it works so there is less reason to care about the weeds. When code is a communication of intent between people, or even just between a person and their future self, learning to encode that intent well into the code takes practice. There are entire fields people have developed over this, like domain-driven design or data-oriented programming, ensuring the data shows what it means semantically.
It is impossible to reconstruct the intent/spec from the code. More often than not the intent is not clear in the code, and one has to track down the original author to check their context and decision at the time.
As the saying goes, naming things well is a hard problem. It's kinda mystical in a way, where only by knowing an entity's true name will one understand it.
However if it's communicated between agents one may care even less.
This is an interesting distinction. I can't stand AI writing but have no problem with AI coding.
I think the key difference is that code is a tool used for a given purpose. It can be slop, or structured better or worse, but at the end of the day it exists to be executed, and its execution is either correct or not. Additionally, good code that does a given thing can only be expressed so many reasonable ways, probably. So LLM-generated code can be perfectly fine.
By contrast, human writing is about exchanging thoughts. Often those thoughts are about infinitely complicated and poorly understood systems, like other people, or emotions, or politics, so it's not a matter of exactly encoding something but attempting to describe it imperfectly. All the tiny little details encoded in how someone writes something can work to this end and are also a product of the writer's thought patterns and overall personality; this all conveys useful information to the reader. In other words, there is a much greater degree of fine detail that affects the quality of writing than of code, and LLMs suck at those fine details for most subjects that are interesting. On top of that, they don't have original thoughts.
That's only one side of the article's point though. The other, more consequential part is:
>Like most writers, I find writing an almost-essential tool for thinking. Often it's only by writing something down, that I can really get my thoughts on a topic into coherence. ... . As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs.
There's an analogy with LLM coding that is so screamingly obvious I'm embarrassed to point it out (changing "writer" for "programmer").
I'm curious to know if the author, given who he is, thought of it (he had to, its just too obvious) and what he thinks of it.
> I'm wary of them, partly due skeptism about their accuracy, but also because it rather misses the point.
Here, this reader wondered whether "to" was omitted intentionally to emphasize lack of LLM involvement. Then, on a second read through to find the sentence again, I noticed the first paragraph is also missing a period on its final sentence. Which just goes to show: we can all use editors and proof readers. (According to the acknowledgements section, nine humans read drafts.) Now, I don't want to intentionally leave typos in my writing, but for me it's preferable to LLM-voice no doubt.
> But I also would prefer rather a genuine human's clunky prose to the smooth but soulless LLM.
Amen. This reminds me of a bit from Kitchen Confidential about how Anthony Bourdain would prefer a simple and plain home cooked meal over an ornate restaurant meal any day of the week. I read because I want to hear what a human has to say. It's the errors and malapropisms and style that give a unique and human voice. Doesn't this contradict my earlier statement about not leaving errors in my own writing? It turns out I'm perfectly tolerant of errors in anyone's writing but my own. (I should probably bring that up with a therapist.)
There's a fine line between using an llm to write a personal blog vs using it to help you come up with an account plan on why you want to sell to a company that will be used internally.
> There are tools out there that claim to be able to tell the difference between LLM-written prose and the old-fashioned biological product. I'm wary of them, partly due skeptism about their accuracy, but also because it rather misses the point
HN uses automatic classifiers. They seem pretty effective. At the same time, I think it's a loosing battle. I'd rather HN either flag for others to ignore/filter/review if they want to, and/or allow people to self-report (ala TikTok)
Eg. I wouldn't mind if a comment on HN was like:
Hey everyone, I coded this little thing:
<ai-generated>
some code
</ai-generated>
Then users can choose if they want to see posts with AI-generated stuff or not, or something else...
Of course there are always going to be some people trying to game the system. But what are we optimizing for? (a lot of times the 95% unnecessarily suffers because of the mitigation efforts to prevent the other 5% from abuse)
i noticed something similar yet different. because, sadly, most of what i read today is llm output (the chain of thoughts, or the summary at the end of a big chunk or work), i tend to "adopt" this style in my writing, hence, writing as an llm without using one. humans are an amazing creatures at copying stuff and adapting to our environment
A billion humans use AI daily, they write about 0.5T tok and read about 2T tok (estimated). In a week they amount to more than the whole training set of GPT-4. Human language is being shaped back by AI language.
My workplace is so AI-pilled now that people have Claude responding to Slack threads on their behalf. It's not even discrete - the messages are literally tagged with a "posted via Claude Code" label, and unabashedly use LLM voice. Sometimes it's unclear to me whether the poster even understands what Claude wrote back for them.
It's infuriating to read, but even more so, it feels disrespectful. It's like they're implicitly saying, "I can't be bothered to respond to you, so just parse this AI slop".
I don't really know how to make a case against it since leadership is very pro-AI usage. Has anyone dealt with something similar and have suggestions?
I think the bit about "saying what you write" is what stands out.
I wonder if you took the pure speech to speech models and STT'd the output if it would remove the LLM voice? After all, can you even imagine hearing a human utter aloud some of the nonsense you see spewed by claude? Or give a longform presentation or speech in the form of the essays that are 100% LLM-written?
Maybe there's something in our human brains connecting oration and writing that influences the style of writing that pure-text engines just lack?
> 3: Michael Nygard commented: “There you really aren't trying to communicate thoughts from one person to other people. You're communicating with a process rather than a human. A human may open the envelope but that person is acting as a component in a cybernetic organism.”
But of course: manufactuting itself creates " productive mechanism whose parts are human beings, as Adam Ferguson describes the industrial workshop: "an engine, the parts of which are men".
I look at it like this: LLMs can generate code for those who know nothing about software development and they can generate text for those who know nothing about writing. Asking them to write it themselves will not get us better writing, but rather the opposite. If they were willing to learn, they would be doing that right now instead of using a tool that absolves them from learning. So, as bad as LLM slop may be, it's quite possibly still better than the alternative.
I never use LLM to write anything, but unfortunately now whenever I am writing something, I am worried, am I sounding like an LLM? Using certain words, makes me worry, it will make my writing like an LLM, so I have started to avoid some words in my writing.
It's a little funny to me to read this from Fowler, given how enthusiastic Uncle Bob has become about agentic coding (albeit his loop specifies very strict code quality metrics). But yeah, I completely agree with the take. Fowler is, as they say, absolutely right.
> (it's not the em-dashes, it's the load-bearing tapestries)
I suddenly feel an urge to turn this into an image prompt.
I used to work on how my writing would come across and on writing what people would like. Now I spend a long time on how to cut the LLM voice out of what AI wrote for me (I keep repeating "this sounds too AI, fix it" lol)
> I'm also suspicious of my own ability to accurately recognize LLM voice.
With much sympathy for the headline this is the central problem.
It's not easy to say what Uncanny Valley looks like, and so we're left with suspicion not certainty.
BTW, I am now 4-5 out of hundreds of my comments since AI detection became a thing, called out for having "AI" tone. I fucking hate being objectified like this. It's not true, although I understand that assertion is hard to refute or support. All I can say is, some people write like some people believe an AI writes.
I'm big into AI but hate watching a youtuber when I suddenly realise their script is pure AI garbage, that they didn't even edit out the 'quietly's or the the kickers.THE KICKER is forever ruined. I hate it. The kicker can quietly get bent.
> In time it may be that LLMs will be so good that we can't tell the difference.
I have to believe that we are within a couple of years if not months of this. This problem is too well-known to not be a major focus of the frontier labs' efforts.
My own feeling is that this signal will perhaps never fully go away. Not in months; most likely not in a couple of years.
Not least because it’s getting worse rather than better. But also because as AI gets better at writing like humans, humans will change how they communicate to avoid being confused with it.
Which will get a lot easier once human writing is nearly extinct. Now that could happen in two years.
Yeah, I swear AI writing is worse than it was. Part of my big model release tests has always been asking it to write a few chapters in different styles, and I swear they have slowly gone from really bad, to boring and plain (I remember Gemini 3 being peak of this), to now over-complicated and quite bad. My guess is that they were initially being trained on writing quality but that has taken a backseat since it's not really a profit generator.
I think the signals also show up in different ways to different people.
For example, I don't dislike the way Gemma 4 writes so much.
But I am British and Deepmind's output somehow seems more compatible with the way British people communicate (it's a very multinational team, I suspect that is an influence).
Other people pick up on Gemma's AI writing tells a little more readily than I do. I have a greater tolerance for Gemma (and Muse Glimmer) than ChatGPT or Claude (or Qwen).
They're still heavily RL'd on response formatting/clarity, and against common LLM-isms. Long form prose quality is expensive to RL so I expect this to languish for a while until other low hanging fruit has been picked.
The trouble is that it's not an intelligence problem. You can make an infinitely capable LLM. It will still have a default voice.
The reason people notice "AI voice" is not that the AI voice is itself remarkable. It's that it appears so frequently. Every literate person spent nearly their entire life reading text written by a variety of different individuals. Dozens of different voices in your morning paper. Hundreds of different voices on every web forum. You get the idea.
Now, imagine that all of those voices were replaced by one guy (or two guys, or three guys if you're a Gemini believer). Everything in the newspaper is written by one guy. Every comment on reddit, and HN, and the NYTimes comment section, all written by the same guy. Other people tell him what they're trying to say, but he's writing all of it. Some people might even ask the guy to write in a different voice, but the voice is just that guy doing an impression of someone else, it's not actually someone else.
That guy's voice would probably start to get pretty grating. You'd start to notice every tell, every frequent word choice, every formatting quirk. You'd start to wonder if that guy was actually a good writer, or if having him write everything was just more convenient for everyone. You might even start writing like him yourself. This would be true even if that guy was an absolute virtuoso of the language and his writing was unassailable in a vacuum.
If everyone is using the same (small set of) model(s), then it doesn't matter if the model is GPT-3 or Claude Legendarium 9. Unless the operator - which in theory doesn't need to be a human - is doing something to move it off of its default voice, the outputs are going to sound the same, and you're eventually going to figure out, and get annoyed by, its tells.
Thank you for posting this.
I don't have a lot to add on the subject, but I found it interesting and thought-provoking given the age we live in.
I find it really hard to tell real human and LLM text apart, even though some people claim they can, calling it "AI slop".
I think the key tell is that the writing doesn't actually illuminate anything worth learning. The more I see of AI produced output, the more I believe that there is nothing there that wasn't already in the prompt. It's an inflation of a single idea to pages of text. It meandors about and never actually gets into detail.
It's a lot like talking to someone who doesn't know what they are talking about but are good at faking it for a while. The game works by leaning on pat phrases, circular reasoning, and hoping nobody is paying too close of attention. I think this is why it's so popular with the corporate executive class, as that has been their mode of communicating their entire careers. To them, they don't see anything wrong.
"Shitty writing existed before AI" is unfortunately an all too common rejoinder to this issue. That's not an excuse. That doesn't make it better. Just because the pointy haired boss talks that way doesn't mean we should be gleefully pouring gasoline on the fire.
I noticed the exact same thing, and I also don't want the LLM to suggest anything to me. I do want it to comment, and to tell me what could be improved, but not to give me text.
I created a small tool to make this easier: https://www.writelucid.cc/. It's free (bring your own API key), feedback would be appreciated.
I disagree with the advice to avoid having an LLM write a first draft and then editing it yourself.
To me, for most business and academic content, the ultimate goal is to convey information. If I can edit AI copy to legibly convey that information clearly, that's enough and I can move on.
The saying, "Don't allow perfection to be the enemy of the good," comes to mind.
And I'm not convinced an individual can accurately edit without learning just as much as if they wrote the content. I guess the question is: how hard does the person work at editing. And does the individual challenge themselves to not put out content they themselves don't understand.
We're in a transition period. And as in all transition periods, many recoil at what is new. LLM writing still feels strange. And it's understandable.
But, for better or worse, in a business and academic setting, as the years go by, most of us will shrug and move on.
As with any topic, there will always be a minority of fanatics who cling to the rejection of AI writing in all its forms. Just as some prefer Vegan leather or tabs over spaces.
But among those who get things done, most (though not all) will accept that it's not worth it to exclude AI from writing. There simply isn't sufficient ROI.
It's like trying to bring back low-end highly human employed manufacturing to wealthy industrialized nations.
That ship has sailed.
I do agree, however, that in cases where an individual is reading for pleasure -- e.g. not in a business or academic context, authors will be forced to pick their audience: those who enjoy writing, even if it smells of AI, and those who don't.
I guess to summarize, my feeling is that using LLMs to write is perfectly acceptable, and used well you'll likely learn just as much. But you must be willing to lose some of your audience. And, over time you'll lose less and less of that audience.
I don't know, with LLM it feels like I finally have possibility to prepare all the dull text that corporate environment expect in a form that colleagues are able to digest even if they don't specially enjoy it like it's the best poetry their beloved one ever composed specially for them. This is in sharp contrast with the struggle to shrink into a somewhat coherent excerpt the wide web of interweaved psychological experience flow graph into this alien expressions which are text streams. It's still impossible to dump really the whole experience and communicate wholly the full semantics they cast, but at least these automatons are never scared to face far much than they can sustain in what can be thrown at them, and apparently most people prefer the dull soulless prose to whatever can suggest vivide continuous reconnection of everything as the context inevitably drift and everything needs to be reconsidered properly with all the rest.
It's incredibly hard to maintain your own voice while using LLMs. It requires deep discussion with the LLM itself so that it comes to understand your perspectives and also explains to you the premises on which its own recommendations are based.
I haven't really done a systematic analysis to compare the quality of fully human-written content and content generated via this kind of deep human-AI collaboration, but my intuition is that the latter is vastly superior. Perhaps AI at its best is a research tool, but a genuinely intelligent one that we need to engage in conversation with in order to use to its full potential.
The sense that I get is that as a conversation gets more in depth, the LLMs start behaving more like reasoning engines and less like statistical averaging machines. The former is vastly more useful to a person who wants to develop an idea and articulate it well.
> But if I want to communicate with humans, my humanity has to shine (or at least ooze) out of my prose.
Yeah. I want to be able to touch a human being (or at least have one touch me) through the words. LLMs poison that, even if a human prompted the LLM.
Translation... I think I object to translation less than Martin does. There's a human back there, and they're writing all the words, and then an LLM is translating them. The LLM may not translate them perfectly, but they aren't going to introduce "load-bearing" when it wasn't there.
The bit about using LLM when you're writing for a bureaucratic process rather than a human was interesting, at least to me. I hadn't thought of it that way before, but I think it makes sense.
People have been talking about the importance of curation for more than a decade.
Because many good ideas are not immediately obvious good ideas, in an environment at risk of information overload, we need some way to quickly vet ideas to see whether they are worth investing the time to consider them. (It is often the case that the best ideas require considerable investment of time and effort to be judged as good, and, it's also often the case that this determination can be made only at the very end of such a process.)
“Don't judge a book by its cover,” sure, but, historically, there have been superficial markers of care and attention that we've used to these ends. These are obviously very flawed signals, but, you know, maybe if you took a lot of time to craft a message, I don't feel that bad if I waste a bunch of time trying to decode it. These markers—attention to detail (e.g., spell and grammar checking,) clear-structure (e.g., “bottom line up front,”) and typographical flourishes (e.g., smart quotes and em-dashes and extended unicode like diæreses…)—were an indication of some modicum of quality control and care. I, myself, used to brag about or show-off having set up .Xcompose keybindings (https://wiki.archlinux.org/title/Xorg/Keyboard_configuration...) which was just rare enough across the general population to be noteworthy.
All of this that was once somewhat expensive is now cheap or basically free.
Even if I make a point of going back to a pre-UTF-8 Usenet style (or even intentionally insert run-on sentences or misspellings or grammatical errors,) the cost of an LLM to style-transfer its output to match this is barely much of anything. All it means is I wrote one prompt for the first draft, and then one more prompt for the refinement.
Additionally, some LLM habits are actually good habits. Putting the bottom-line up front (in bold, even!) is a great habit in a business e-mail! Decision-makers genuinely struggle with reading through long paragraphs of detail!
Unfortunately, the problem with the LLM output is that it takes good, valuable, superficial or structural habits that it can cheaply apply, and it drapes those around inane, banal, cloying, muddy, meandering meaninglessness.
Today, we can maybe still judge a book by its cover, because even one more prompt worth of refinement is still too much effort, but I don't think that's amounts to much, since the frequency and loudness of complaints of LLM writing means that model trainers are going to train their models to adapt to whatever signals we use for quick vetting. I am skeptical whether this training will concurrently improve the novelty, innovativeness, or other intangible qualities of the actual message. (With the models, as they exist today, and as we use them today, at least in the field of software development, I see extremely poor capability at generating innovation… Personally, I don't even like the quality of the Python code they spit out, but they sure do type real fast, and, like, pejoratives aside, unexciting, mediocre, corporate-y code does have a real place in the world.)
So what happens when the economics of production in the medium significantly outpace the capabilities of production in the message? The LLM can write a comment nearly indistinguishable from any given human… except when you actually take the time to read through it, it's just inane drivel. Thoughtless and a waste-of-time. And there's basically no way to manage your investment of time and effort to tell one way or the other.
You know, maybe it was never the case that superficial markers of effort ever meant anything at all. Maybe there never was a way to tell from the envelop whether the letter is worth reading… and it was confirmation bias all along. Maybe there really are no shortcuts to judge whether an idea has any merit…
Because, for sure, some of the real bangers of comments I've posted on HN and similar forums—all my XCompose-configuration aside—have gotten relatively few upvotes!
An important distinction as AI-generated content becomes more common: having a consistent voice is not the same as having a distinctive point of view. The best AI-assisted writing should preserve the author's personality, judgment, and expertise rather than simply imitate their vocabulary or sentence structure. This is a useful concept for anyone building content workflows with LLMs.
Great piece. The "LLM Voice" problem is real — we've been benchmarking 18 free LLM API tiers (Qwen3.8-Max, GLM-Z1-9B, Hunyuan-MT-7B, bge-m3 embeddings, etc.) for consistency and tone variation across providers, and the homogenization is striking. Models from different families converge on similar cadences and filler patterns even at different parameter counts. The one outlier we found: reasoning-focused models like GLM-Z1-9B actually break the mold with chain-of-thought traces that feel more distinct. We publish the live benchmark data at apishare.cc/leaderboard if anyone wants to compare latency/quality tradeoffs across free tiers. Would be curious if others have found specific model families that maintain a more distinctive voice.
personally, i'm for LLM and AI usage in writing if it helps the writer communicate ideas, their stories, and triumphs in ways they may never have been able to before. I just had one of my posts removed from the front page here which was disheartening because it feels as though what was judged was prose rather than the technical insight in to a niche problem, which seems counterintuitive considering i've always thought of Hacker News being an inclusive place for the technical and engineering skills rather than their level of the english language.
> I just had one of my posts removed from the front page here which was disheartening because it feels as though what was judged was prose rather than the technical insight in to a niche problem, which seems counterintuitive considering i've always thought of Hacker News being an inclusive place for the technical and engineering skills rather than their level of the english language.
The problem with LLM voice is that it sends the signal that it's a likely waste of effort to try to figure out if there's any "technical insight" there, because more often than not, there isn't.
Maybe it's not fair that you're paying the price for others using LLMs to abuse their readers, but it's the reality that you have to deal with. It's also not fair to expect readers to wade through all that abusive LLM prose in order to find your (assumed) non-abusive post.
Once LLM writing becomes indistinguishable from human writing, it will be really sad. There will just be a collapse of human engagement, and you might as well just talk to your machine because then at least you know what you're getting and aren't being deceived.
in some ways I agree, however I think that speaks more about the reader themselves. Just like when communicating with someone who's first language isn't english, I don't disregard the meaning and intent behind their words simply because the way they communicate isn't formally correct. Regarding my post that was removed, it was lovely to see the comments of people that had enjoyed the technical write up. Me personally, i'll always value an attempt at engaging and communicating than none at all.
Good lord, someone not being good at english does NOT mean that they put not effort or care into their attempted output. It is RACIST to make that association and thats why its an obvious faux pas.
An LLM prose article ABSOLUTELY can mean that. To attempt to conflate the two things is maliciously misleading.
This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.
Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
> This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.
I think you're misunderstanding that anxiety. I think this is what's going on:
1. For a lot of people, using technical skills like coding is the part of the job they enjoyed.
2. Closely related to the above point: for a lot of people, technical skills was the part they were good at.
> Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
> But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
3. More broadly: if "they" don't find the "technical skills" you provided valuable anymore, why do they need to keep you around? Why not cut your 10-person team down to 3 people (the manager, architect, and technical lead) and fire you and the other 6? Not everyone can re-invent themselves as a product person, even if they had the skills. It's a game of musical chairs.
No I get it, I didn't mean to be dismissive sorry.
It really is ultimately a labor relations problem. A lot of people thought they were guaranteed to not face penury and now they're realizing they may have been wrong about that. The issue is that it's ultimately not under your control whether you can sell your labor, and there's no universal fallback if you can't. Solidarity and concrete power for workers are the answer to this and I don't think there's another one.
For the rest, yes, certainly. The vast majority of people do not enjoy their work, are not valued or rewarded for the skills they apply to it. Again, if you thought your job was writing code this sucks. But you're merely one of the billions now. I understand the grief for what people are losing but very very few people ever had that rare privilege.
I find a lot of LLM writing exhausting to read. It is like I have to exert extra energy to attenuate my BS detector to cut through what is actually being said, and why.
Imagine you cooked something great, a deliciously tender cake or something. I ask you if I can try some, you say sure, cut off a little piece for me, then you drop it and tell I can have it on the floor. That is what AI is doing for writing. Why should I care how the cake would taste, if I don't like eating on the floor?
i don't really agree with that view, but i respect your opinion. In my view, i see it more like someone who's never baked a cake before, offering you a slice, and whether or not it's good may be up to you to decide.
AI doesn't help the writer communicate ideas. It helps them obfuscate those ideas behind bloated language where even the writer can't discern if it's good or not.
It's the modern equivalent of going to a thesaurus and replacing all your natural words with five-dollar ones.
If you're not confident in your English then just write your brilliant ideas in broken English. Or at most do a direct translation. But don't fluff it with AI's nonsense.
Assuming the author had to input good points for the LLM to expand it into a full length post, just give me the points. Give both formats if one must. Translate, spellcheck, fix mistakes with an LLM. Don’t inflate 3 bullet points to a 10 minute read.
I can’t distinguish between well meaning slop and blogspam slop. I might be able to if I’m well informed on the topic and can point out inaccuracies. Otherwise I just don’t know what is accurate and what is filler or outright wrong. LLM-isms are very distracting and exhausting to sort through. It just doesn’t make for a good reading experience.
Yes, this. I mean, if you're writing a novel meant to be read, maybe not. But if you're just trying to communicate 3 ideas, write them out in a way I can quickly tell if it's worth reading more.
There are few things (relatively speaking, I mean online activities, etc) more annoying than seeing a click-baity headline, and then having to read 17 paragraphs to actually get to the point that I realize I already knew and understood it, and it's just slop padding.
I'll repost a comment I made a while back describing the process I use, which I think balances AI in keeping your voice.
If you care about your voice, don't let LLMs write your words. But that doesn't mean you can't use AI to think, critique and draft lots of words for you. It depends on what purpose you're writing it for. If you're writing an impersonal document, like a design document, briefing, etc then who cares. In many cases (scientific papers, legal documents) you already have to write them in a voice that is not your own. Go ahead and write these in AI. But if you're trying to say something more personal then the words should be your own, AI will always try to 'smooth' out your voice, and if you care about it, you gotta write it yourself.
Now, how do you use AI effectively and still retain your voice? Here's one technique that works well: start with a voice memo, just record yourself maybe during a walk, and talk about a subject you want, free form, skip around jump sentences, just get it all out of your brain. Then open up a chat, add the recording or transcript, clearly state your intent in one sentence and ask the AI to consider your thoughts, your intent and ask clarifying questions. Like, what does the AI not understand about how your thoughts support the clearly stated intent of what you want to say. That'll produce a first draft, which will be bad. Then tell the AI all the things that don't make sense to you, that you don't like, just comment on the whole doc, get a second draft. Ask the AI if it has more questions for you, you can use live chat to make this conversation go smoother as well, when the AI is asking you questions, you can talk freely by voice. Repeat this one or two more times, and a much finer draft will take shape that is closer to what you want to say. During this drafting state, the AI will always try to smooth or average out your ideas, so it is important to keep pointing out all the ways in which it is wrong.
This process will help you with all the thinking involved being more up-front. Once you're read and critiqued several drafts, all your ideas will be much more clear. Then, sit down and write your own words from scratch, they will come much easier after all your thoughts have been exercised during the drafting process.
As someone who is aggressively pro AI, I've also noticed how awful everything is to read lately. It's always so hard to put my finger on the exact thing, but it's a mixture of everything sounding identical and everything being a rambling unconcise mess. I find I routinely start reading interesting posts and just .. stop part way, abandoning it. A site I read stories on has been flooded with AI generated content, all of it terrible, and it just makes me retreat to things created before 2023 as a way of filtering it all out.
A model trained to minimize statistical prediction errors is going to sound ... predictable (aka boring). A (good) human author tries to keep their writing interesting, while an LLM strives for predictable instead.
It's not just predicting content but also the form/shape of the response, highly shaped by RLHF preferences, which seems to encourage longer form responses which need padding with filler to generate the "preferred" amount of verbiage.
It seems most of these models have been trained to interpret questions as wanting a long-form authoritative/comprehensive-sounding response (even when midwit filler is all it has to offer), rather than initiating a short-turn conversation which would better allow you to steer the conversation. Apparently Muse is much better in this regard, but so far I've resisted signing up for any Meta/FaceBook service, and not inclined to break this winning streak!
Human writers also use specific constructions for definite (though not always consciously known) reasons, and mapping those correlations onto writing models is only getting done on accident. This is a big part of why LLMs doing pointless negations (it's not X, it's Y) is so jarring: people use that construction in order to preempt an expected misconception, whereas LLMs try to use linear algebra over the learning corpus to generate it in appropriate quantities and places according to RLHF/fine-tuning/etc.
In other words, it isn't just verbiage that has a "preferred" quantity, but also higher-level language features, and the appropriate contexts for those features feel like the sort of thing that will be extraordinarily difficult to replicate without a theory of mind.
edit after coming back and re-reading: the "in other words," and rephrase is because I'm trying to explain something complex enough that I'm not even sure I fully understand it, so I try typing out multiple explanations and leave the ones that make the most sense to me. This might be idiosyncratic to my writing and not a significant feature of training corpuses, but even if it wasn't it's a second example of the sort of thing I'd suspect LLMs to be bad at using appropriately.
Another LLM weirdness is to explain what code is not doing. My theory: such negations were somehow encouraged in training because they are very valuable where used properly (and sparely) by humans. "Not doing the obvious thing due to xyz happening two modules over" is a very useful comment. "Not using bubblesort because quicksort is faster" is garbage.
I think a lot of the time it's prompt leakage. Forgive the personification, but it's like they don't have a mental model of the reader of the thing that they're authoring being separate from the person who's prompting them to author it, so it often seems like the main conversation kind of spills into the prose.
Not to pick on GenHTTP Lambda, especially because I think it's someone's fun side project, but I was just trying to read though their docs and I was finding it super difficult. I kept getting distracted thinking "why the hell did it write it _that_ way?"
Like, the first sentence of "The control center" section [1] is the exact sort of weird negation that you're describing:
> The editor link opens a control center rather than a text box: most of the code here is written by agents, so the first thing on the screen is how your lambda is doing.
I can't help but think it generated the first round of the page with a text box to paste the code into, got told "no, I want a control center," and then this ended up in the docs.
I also think the phrasing of that sentence (and many others in those docs) is just super weird and unnatural - e.g. "so the first thing on the screen is how your lambda is doing." It also feels like it mirrors a prompt - like, "no, it should be a control center, not a text box, and the first thing the user should see is how their lambda is doing." But in docs I'd never expect it to be written "the status of your lambda" or "the health of your lambda." And what does the code being written by agents have to do with it?
1: https://genhttp.dev/en/docs#editor
Lambda is my side project and I 100% agree. Updating documentation with Claude does not work, unless you spend so much effort that you could have written a perfect version yourself. Opus 5.5 got better at it, but it's still a long way to go. I personally cannot stand the PR descriptions generated by Claude. Three pages of text for just a simple change ...
This is my experience as well. I have spent many hours trying to eliminate this kind of contamination and the only solution I've found is to re-write from scratch every time. If the result isn't to your liking just reset its context and provide a more detailed prompt.
1000% this. I encounter this all the time (with Claude) and it drives me nuts!
I vaguely remember a study that I was able to find when fact-checking this comment before publishing it [1], which found that music needs periods of high predictability to nurture listener expectations for the sake of later subverting those expectations, and the most enjoyable parts of music are those moments where expectations are subverted. This sounds like a similar effect.
1: PDF warning https://www.cell.com/current-biology/pdfExtended/S0960-9822(...
uh huh. so music is like a joke - the payoff is unexpected, while simultaneously adhering to scenario expectations.
Yes, with the "scenario expectations" just being the necessary setup to deliberately create false expectations and make the punchline that much more unexpected/funny.
I think both music and humor are both exploiting the same evolutionary essentials of human nature - to find comfort/safety in predictability, and excitement/danger in unpredictability. Laughter would seem to be a social thing - perhaps it helps develop a theory of mind in signalling that you get the joke - that the punchline is indeed unexpected to you as well ? And/or perhaps it helps draw others attention to an unexpected situation with learning potential.
Gemini models are so, so much better in this regard. Their outputs are actually readable.
Agreed. In my experience it gets to the heart of information faster rather than so much filler. Also better information and more comprehensive in general.
The main issues seem to be glazing that sibling mentions and also it seems to play a bit loser with facts and reality and makes up more stuff.
Gemini tends to glaze you though. I now believe I’m a genius at pretty much everything after trying it a little.
Every model has done this. And sometimes the makers go the other way to compensate, and then you get Opus 5 who answered like it was possessed by the ghost of stack overflow.
Not really, Gemini is at a different level, it will naturally burst out compliments and honorable nickname to provide emptional values. My last company was iterally destroyed because the old boss (successful boomer who had no idea how software industry work) believe everything his Gemini told him and gives us all kinds of unrealistic assignments and force us to use whatever tools/services Gemini suggested.
"A model trained to minimize statistical prediction errors is going to sound ... predictable (aka boring)." -- what if the goal is to be a (good) human author.
In that case, model will in fact strive to keep their writing as close to interesting writing as possible.
The goal (literal, not figurative) of an LLM is to reduce training/inference errors - to predict the training material.
The "quality/goodness" (likability) of output is only selected for to extent that it also reflects RLHF preferences originating from low paid contractors who are scoring based on "looks good to me", not based on Pulitzer prize prose potential.
To explore a false hypothetical, suppose an LLM was only trained on the output of a single popular author, then you'd certainly expect it to reproduce that author's style, but it would not be "upredictably interesting" but rather predictably quirky, reproducing the same "surprises" over and over so that they no longer are surprising and therefore interesting.
To continue the hypothetical, if you trained an LLM exclusively on the output of multiple good authors, then rather than get the best of each you'd get a statistical blend (this is how training works) that emphasized what they had in common rather than the individual quirks that made them interesting.
There is a fairly common misperception that "AI" (LLMs) is always trying to do it's best, where "best" has been anthropomorphized to have the human sense rather than the LLM sense of "the most statistically accurate continuation prediction" it can make. For example, if you give an LLM the moves of a partial chess game and ask it to predict next moves for one of the players, it will NOT play to win, but will rather play to predict how the game would have continued, which means continuing at the level (ELO rating) and style of the player who had made those moves.
Of course you can ask an LLM to play to win, just as you can ask it to write in a style that is, say, "unpredictable and interesting", but it can't escape itself - you will necessarily get predictable unpredictability.
I think you're missing what I'm saying; the LLM can and will have trained on all interesting prose, and it will evaluate all interesting prose. If you indicate that you are goal setting for amazing prose, and you give enough chain of thought, it will self evaluate.
Your chess game example misses the point. If you give an LLM the moves of a partial chess game and ask it to predict the moves of the players, it will predict how the game would have continued, even if it is low level players. But if you ask it how Magnus Carlsen would win the game, it will likely come up with a sequence that can beat most human players, or possibly just decide to invoke a chess solver, which will most definitely win.
The key for your prompts and thinking loops is you need to indicate proper goals. "Write the a story about 1775 England." is different then setting a goal for it to write the a fantastic storyin the style of Dickens about England in 1775.
Regarding the ML portion; what you've written about a 'statistical blend' is a little bit coarse. That would be what you might have gotten from a 2 layer neural net or a regression in 1999; the neural networks today operating with attention or an incredibly high number of parameters are non-linear and non-interpretable, that's the entire point. This isn't a regression of simple parameters, it's all interactions of all parameters deeply netted. Model interpretability vs. Model Prediction power are generally inversely related.
The model can in fact escape itself in that it has many, many, many more parameters than a simple "predictable unpredictability" and that is exactly the point of it.
I'm not sure how you are imagining an LLM "self evaluating". They are feed-forward auto-regressive predictors. The only source for the next word they spit out is what came before and the patterns that contains.
If you ask an LLM to "win like Magnus" it can't because, as above, it is a feed-forward auto-regressive predictor. Do you realize how much different this is to how something like Stockfish (that can play at, and far beyond, Magnus' level) works?
A big LLM is still an LLM - the architecture and operation doesn't change. It just has more capacity to learn, both to learn deep predictive patterns and to memorize factual knowledge.
Does it not occur to you that your beliefs of what LLMs are capable of (generating amazing prose) is totally at odds with what we see them doing? Where is all the amazing prose?
What if? But this certainly doesn't describe LLMs.
Interesting, I was thinking the walls of text are better for the watermarks.
I'd be shocked if it wasn't a shotgun strategy learned during RLHF or RLVR post-training.
A short response that's wrong is going to be punished, whereas a long answer that contains the correct information, and then a whole bunch of other stuff surrounding that info, is more likely to be rewarded. Even if you discount the reward by answer length, when there's ambiguity involved it's still going to be a better strategy on average to hedge by saying too much vs risking saying too little.
> It's not just predicting content but also the form/shape of the response
Nicely done.
That doesn't follow. Predictability is your brain reacting to the lack of novelty. What an LLM produces is a chain of words following some kind of conditional distribution, whose output could vary significantly from your understanding of novelty depending on its training and priors.
With a single, generalized prompt ("produce something unpredictable"), ChatGPT responded with:
> 00:— / ANOMALOUS RESIDUE > Vekh. > Ìssara. > Nnnn—tq—èl. > > A temporary thickness enters the meat of the signal. > > Not rain. Not a substance with permission to fall. > > UL / 9 / ferrous / ∂ > The measurements are eating each other. > > Someone has left an entire childhood in the wrong unit of pressure.
That is not predictable to me, at least. It could be boring, but I think that is mainly a product of me knowing where the text is from. LLM-generated text is inherently boring, to me, but that has nothing to do with predictability.
Don't these sort of conflict with being "aggressively pro AI"?
> how awful everything is to read lately > flooded with AI generated content, all of it terrible > everything sounding identical and everything being a rambling unconcise mess.
I can see the benefits of AI and enjoy using the technology - but, like many have said, it's wreaking havoc on the "content" that's out there nowadays. Gotta take some bad with the good, I suppose.
> Gotta take some bad with the good, I suppose.
Well, it's quite simple if you think about it. Intelligence is all about awareness of the imminence of death and the finite amount of time given to any of us. So, we humans intrinsically evolved to make the most of it. The majority of the tools we create have the ultimate purpose - to save time. Even money is nothing but a way to cut some time. It's not really about time but experiences - we want to have as many and as rich as we can get, and we innately try to cut time somewhere else, so we can use what we saved for better experiences.
Therefore, inventing cars was an easily predictable event. There's not a single hypothetical timeline or dimension where humans don't end up with cars - to transport things and to travel faster. Similarly, it was always simply inevitable - we would have invented something like LLMs, sooner or later.
Cars required significant sacrifices - they are absolutely deadly machines, they kill thousands of people every single day, kill wildlife, endager our own existence, and we're kind of okay with that. I can absolutely assure you, if someone tomorrow invents a gravitational jump travel that allows moving things between continents in minutes, even with a success rate as low as 10%, we would still be using it.
So, yes. We can't un-invent cars and AI, the only way to move forward is to adjust to this new reality and extract the best possible experiences, preferably by not robbing other people of their - i.e., enjoy life while it lasts, it still remains finite.
> Intelligence is all about awareness of the imminence of death and the finite amount of time given to any of us.
You lost me there. Children are pretty intelligent and have no concept of time. I don't see the relation you find obvious and the rest of your points similarly don't follow.
> Therefore, inventing cars was an easily predictable event.
This feels like a benefit of hindsight unless you can call out other obvious and imminent (but not yet existing) inventions.
Yeah, I guess, "intelligence is all about awareness of death" is kind of a bomb of a premise. I admit, not the best thought I spat out through my keyboard. I suppose I mean some narrower sense of intelligence here. On one hand, you're absolutely right - crows, octopuses and toddlers solve problems without any hint of understanding mortality. Yet "children have no concept of time" is incorrect: not only do they have one, but most kids also grasp that death is irreversible and happens to everyone by about age 5-7.
I don't blame you for immediately rejecting the rest of it. But do hear me out: the point about the inevitability of invention does have real support. Historians call it multiple discovery - tons of things have been invented independently - calculus, the telephone, the jet engine, to name a few.
Neural nets, GPUs, and the scale of Internet text made something like LLMs close to inevitable. Of course, we could debate forever whether any of these on its own was ever inevitable. The point is - society tolerates any cost of a useful invention for the benefit of saving time. So it's reasonable to assume this will be the case with AI as well, which is really happening as we speak.
Sometimes less is more
Ironically, this is the case where "nothing is better than more", because I have no idea of what property or context of any of these three comments above you're referring to.
> Don't these sort of conflict with being "aggressively pro AI"?
I don't think so. AI is broad, and he's against low quality output enabled by people taking shortcuts using AI.
Using AI doesn't make your output worse, that's almost entirely down to lazyness and a lack of skill. Or people are just choosing to spend less time on the same piece of work instead of doing additional iteration and experimentation (which AI makes a lot more doable).
I can like the concept of the engine, and the leverage it provides, but I hate traffic and get anxious about having an accident on the freeway. And the people behind the industrial revolution have destroyed the climate.
I love computers and the internet, and the internet has ruined a lot of people and things around me.
It's possible to be interested in a new technology and enjoy its leverage while still being extremely apprehensive about how it intersects with game theory and politics. The race to the bottom of the barrel keeps coming for the jugular of every cool new thing.
I’ve been starting to think that a lot of the variance in how people see the world can be explained by whether their attention is drawn to first-order localized effects, or nth-order distributed effects. This is not the same thing as optimism vs. pessimism, but it sure is correlated!
This sounds intriguing. Can you expand on what your thinking is or link to somewhere I could read more?
Or just short term gains versus long term pains (or vice versa, depending on how you view it)
I feel like in my life all I ever see is people mortgaging the future for today though. Hey how about that record storm in California?
It's a multi-mandate to focus on what's best for your yourself and society such as your mind, your body, your chosen family, your friends, your neighborhood/community, your city, state, country, the global population, and more abstract things such as the human condition, when evaluating things such as privilege or fairness or duty.
You often have to make compromises somewhere in the hierarchy, especially in late-stage capitalism where even access to basic needs comes at a moral cost, and as such those compromises sometimes come with a moral duty. I am writing this via a computer built from limited natural resources, some of which have been obtained through what I see as slave labor. I don't eat meat and still have to acknowledge the harm that clearing and maintaining farmland does to wildlife, even if the result could theoretically be a net gain.
When you start viewing situations through the lens of game theory, you realise some compromises are inevitable for the rational individual. But you must acknowledge the individual is not the only concern. As such, I use AI as part of my work, but I opposed the local buildout of a large data center in my community and tried in vain to make the person responsible for the deal understand the issues it could introduce. This isn't NIMBYism, we need a halt on all buildouts until fundamental environmental and accountability concerns are addressed.
And many other aspects of my existence and quality of life, poor and underprivileged as I was growing up, have depended upon US imperialism. I have also produced a ton of waste just existing and owning pets. It's abhorrent.
I continue to ask myself how I can fulfill the moral duty which I have accrued and reduce my dependence on exploitative economic systems, but I also recognize that AI is the new social leverage and we only have so much time to use it to our advantage before it becomes institutionalized and wielded against the people. I also might only find future income stability by evolving with my industry.
Life is complicated.
Well, we have a good understanding of the benefits that the industrial revolution and the computing revolution have given us.
The AI situation is different. The technology is obviously extraordinarily powerful but the benefits to humanity are still nebulous and there are many very obvious harms like the one we're discussing. So to describe yourself as "aggressively pro AI" you must believe that future, as-yet unrealized benefits will outweigh the harms. I don't think that's a crazy thing to believe but it deserves scrutiny and demands elaboration.
> it deserves scrutiny and demands elaboration.
These challenges never go anywhere because it’s essentially a religious conviction. We’re likely in sort of uncharted territory on a macro scale and I reckon people a) either acknowledge that or refuse, and b), either trust or distrust the tech world’s stewardship of this technology. There’s a hundred bazillion data points, and I’m pretty pessimistic about the tech scene at large right now, but I think reasonable people can disagree on this without being factually or logically wrong. But true zealots in the midst of such uncertainty are driven by faith, which can certainly be deftly swayed over time, but definitely can’t be conquered in a debate.
Eh, I think many of the harms of AI are currently also nebulous. It's not as though the harms are all concrete and massive and the benefits are vague and "in the future"
I agree but I see a lot of very concrete harms too. It's been catastrophic for schools, the sloppification of the web has been largely a disaster so far, it's been a disaster for cybersecurity, it's wreaked havoc on the consumer electronics market, it's driven a lot of financial and existential insecurity for students and early career people across various roles, and while I'm not as militant about data centers as some I think their proliferation has been a net negative environmentally (including increased reliance on coal).
This resonates with me and I suspect a lot of others here. As always life is more nuanced that many would have argue.
I'm kind of pro AI while also being pro LESS AI.
These are ultimately all software systems where updates can be pushed out with the press of a button. We just found an amazing new tool in the LLM and people are trying out all kinds of ways of using it, many of them dumb or stupid, this is discovery that has to be done, we'll keep doing it and pushing updates.
It doesn't take rose colored glasses really to infer that over time we'll cull the dumbest uses and extrapolate on the best ones. There's a huge difference between using a generic chatbot, even one driven by a SOTA model, vs targeting a specific business task with good guardrails, skills, and tool interfaces that call out to real business logic when the model isn't the right thing. The latter will emerge more and more over time.
You can be pro-"thinking AI is going to get really good and solve many problems" (i.e. be the sort of person who would "go long" on AI), while also being anti-"applying AI right now to domains outside of the few it's been specifically optimized for [e.g. coding]" (because in these other domains, AI is yet to approach, let alone surpass, "authentic quality" equivalent to that of a skilled junior professional.)
Only a programmer or a business guy would think just because it becomes capable we would want to see AI produce "content". The effect is the erasure of human culture.
This is somewhat of a dramatic statement. What, exactly, is the mechanism by which human culture is erased because LLMs write some stories or compose some music?
The flood of LLM-produced stories and music. Spam killed email.
> Spam killed email.
Sure, but it didn't kill human culture. As writing didn't either. Nor will AI do it. Technology changes, human culture does the same.
The most powerful people in silicon valley are openly aiming to replace the entire species with robots in space. This is their genocidal aim. The fact that bad things haven't happened yet doesn't mean that directionally, they aren't attempting to replace as much human culture as possible.
It sounds silly! That's because they are completely divorced from the human experience, and they have enough power to really make us all suffer.
I think there could be room for something like how holodeck programs are shown to be authored in Star Trek, i.e.: the author does all the primary work, and then the system, if a particular work's design allows it, extrapolates from that to fill in the blanks when somebody decides to make Moriarty super-intelligent or put a hundred androids in a Wild West scenario or take part in a go-karting death race with Frankenstein's monster.
But, that comes with with the same kind of cultural acknowledgement, as seen with the characters in Star Trek, that such algorithmically generated material is an experimental diversion, and inherently not worth the same respect or focus as experiencing something authored with intentionality by a person.
Right, deployed thoughtfully, in a way that improves the human condition (especially aboard a ship far from civilization!) there are uses of course. When everyone is offloading their thinking and writing this is effectively erasing the production of culture.
I'm sometimes those things. I'm more often a writer and a futurist. I am enough of a writer specifically, to really not want to live in a world where AI-generated prose substitutes for human stories and essays; and I feel empathy toward other creative professions (most of my friends are visual artists) and don't wish to see that kind of thing happen there either. I do think we have to push back on things like that happening. And I do think that people trying to push toward that end for monetary gain are erasing much of the joy of being human.
But that being said. I don't think that our need to push back on that particular use of AI, has anything to do with the need to push back on AI itself. Because that's not the only, or to me even the main, point of (generative) AI.
(It's just that "generating wholesale slop that people end up seeing" is the usage-profile that the AI companies can most-easily intermediate and make money on; and it's also the usage-profile that makes it most obvious that "AI is happening", and so markets AI to people not already using AI. So this is the usage profile that all the AI companies benefit from everyone focusing the conversaton around AI on!)
I think a lot of people have a failure of imagination when they think about the upside of "good" generative AI. Which to me is, specifically, the places where generative AI can be better put to use as a tool in a workflow for human creative labor to pass through, rather than being something that substitutes for said human labor.
The programming example might be the most obvious to people here: generative AI lets you write e.g. a hairy SQL statement in the form of a very long natural-language description of the query, or an example of how you'd do it in an actual imperative/functional programming language, and have that transformed into SQL grammar. So you don't have to try (and fail) to remember the arcana of SQL (e.g. how to use things like correlated subqueries, or window functions with explicit windows, or CUBE), always eventually resorting to looking it up anyway. (This is really a condemnation of SQL for having specific one-off syntax sugar for every single thing it wants to encode, rather than a uniform syntax that can fit in your head / trigger IDE auto-suggest.)
But consider Photoshop's "generative fill" tool. Yes, if you're selecting entire huge regions of the image and running that tool on them, it's going to be "doing original work" and the results will be awful. But that's not the use-case the tool was designed for. It was designed to be a better version of content-aware fill, i.e. PatchMatch. If you select a dust speck or tear on a scan of an old photo and run the generative-fill tool over it, the result will usually just be a better version of what you'd have got by running the content-aware fill tool over it. (And if it's not, you don't need to try to fight the tool; you just drop back to using the other tools in the toolbox. Not every tool is applicable in every situation, and that's okay.)
Or consider that newer versions of Melodyne use generative-AI models to better separate audio stems into their own tracks. The input here is your own manually-performed track. The output here is also your own track. Despite being a generative model, the AI here is only generating an internal byproduct that gets discarded (hypotheses for timbre- and acoustic-matched reference instrument performances) to aid the classifier and reduce stemming error/residue.
(And apparently, the compressed-sensing models used by MRI scanners are getting approximately the same benefit by approximately the same means. Less familiar with this, but it's probably the most important benefit here since improving the fidelity of MRI reconstruction means doctors can be more sure that an opacity is really "something" rather than an artifact, and so will have more confidence to diagnose real problems earlier.)
Or consider generative AI used for automatic human-language translation (where it captures much more of the speaker's original meaning and nuance than previous translation models, at the expense of giving the result "LLM voice.") As AI improves at this task, presumably there will be less "LLM voice" and the result will just sound like what the person themselves would have said if they were bilingual, because... that's the task.
In none of these cases is generative AI "producing content." In these cases, generative AI is subsuming the role not of human labor, but of tools that were already used: of "by-the-book" approaches, of heuristic algorithms, of weaker ML models relied upon by heuristic algorithms, etc. But all of these cases benefit from generative AI being improved.
And, to restate my premise: there are many tasks just like these, where generative AI currently has no place (because replacing or "enhancing" any tool in the task's workflow with gAI would currently make the results worse), where you can expect to see gAI eventually replace or "enhance" some tool in the workflow, once gAI model companies turn their attention and benchmarks to trying to optimize for performance in those sorts of tasks.
[dead]
Agree but also want to point out that this started with the SEO boom of the last decade where sites started rambling so they would keep you on the site longer and get more SEO "points", they were articles meant for a robot to catch humans.
AI has made it even worse and lots of content on the web, for both these reasons, is not worth you time (now wonder users don't read most of the text on our apps!)
In a prior evolution of this, if you look into the Get Rich Quick marketing copy of the early 2000s (before SEO had taken over content formatting), you can see a parallel of sorts in those marketers doing human sales optimization Impact with bold and underline. It derives from research into the human response to ads from decades of print ads and centuries of printed matter prior, and I think there is a direct lineage from ‘how can we bias humans through restructuring text’ to ‘how can we bias machines through restructured text’. This isn’t to disagree, but to call out how rapidly humanity (merely twenty years!) has switched from marketing to people, to marketing to algorithms, in search of a favorable bias.
Ah yes I remember meeting this end of spontaneous expression to bend toward whatever might please some algorithm.
Soulless Enshitificatoon Oblivion now exercises a huge weight on societies to such an extent, it's ridiculous.
I suspect it’s partially an average / regression to mean.
If you built a model only based on long form journalism and The New Yorker voice, including both the template structure of the article and the grammar and syntax, the model would replicate the patterns it is built upon. You could do the same for high vocabulary rap.
When you stuff all of English or all of human thought into it with somewhat equal weight, including very technical writing the sum is a lack of style and elegance and personality.
The second problem is that writing often doesn’t happen linearly. You write bookends and fill in, edit and revise. You sort of know where you’re going and can then make decisions how to get there. Language models may benefit from being redesigned to perform passes. Select a template of some sort, fill in the mad lib, then run a detail pass, then run a persuasion / emotion pass, then run a style pass. Current model writing very much sounds like I would expect autocomplete to.
It’s heavily RL’d and it’s gotten worse because they’re using LLM judges, that’s why it’s gotten worse. The last good writing model was GPT 4.5.
I suspect they’re leaning heavily on some kind of auto verifier, and that’s why the writing sucks, because good writing is so subjective.
And they’re probably nerfing what it can get from training as they’ve lost lawsuits for scraping copyrighted work, so they’ll purposefully avoid anything that references a specific writers style, etc.
Isn't this easy for frontier 'labs' to fix? If the user requests lots of writing and prose, then use an auxiliary model which has been trained on 'award-winning' writing instead of all the crap on the internet?
Award-winning writing has all kinds of different styles. The average of them will probably get you some boring median soup once again, though maybe a bit less bland than the LLMs trained on oceans of LinkedIn and Reddit slop.
Even if it doesn't, the effect of so much writing getting pushed out the door through default LLM settings is going to make it feel same-y and bland in short order.
Oh boy, more IP theft and copyright infringement!
The "regression to the mean" or "it's just the training data" hypothesis doesn't work. Pretrained models do not have these weird writing styles and sound much more human. It's only after instruction tuning -- SFT, DPO, etc. -- that LLMs develop weird writing styles.
That is, the original training on massive text corpora seems to produce an average of human writing, but that average looks pretty average. It doesn't feel like LLM slop. But the instruction tuning on specific tasks (answer this prompt, summarize this text, do this math problem), which turns text predictors into chatbots, seems to create strange style features as a side effect. And because that tuning step depends on company-specific training data (rather than just a crawl of the entire Internet), each company's LLM has a different style.
It's not that difficult to measure this. Linguists have had measures of style variation for a long time, though NLP and AI researchers seem to have forgotten. We've generated parallel human and AI corpora to measure it, and can show specifically which writing features change: https://arxiv.org/abs/2410.16107
Currently we're working on tweaking the instruction tuning process to see if we can pin down which steps and what types of tasks tend to produce the weird style features.
I didn’t really specify whether the regression to the mean was from data or tuning. It’s just happening. They are creating neutral average language instead of varied expressive.
The part I find grating is the attempts to punch everything up. ("It's not X. It's Y"). Like, its a react badge component, it doesn't need punching up. You didn't find the smoking gun in a murder mystery, just a misplaced CSS rule.
I've noticed this as well, I've been comparing it to 4Chan Greentext memes. Everything is a short punchy sentence, and every sentence has the same "weight" to it. There's no flow or rhythm, it's like reading off bullet points.
I call it uncanny valley effect for text [Ed: ops use of prose, rather than text is better formulated]. It's a little like with AI images - they started out with six fingers, but now they look "ok" - but slightly off.
Reading some AI text feels like looking at a Michael Murphy sculpture - makes perfect sense, until you start to look closely at the text, and gaps appear - and you start to realize there's no there, there.
https://dustyoldthing.com/incredible-illusion-sculptures/
My model is about gaps - all ideas start with gaps. LLMs can fill in these gaps quickly - but some ideas are just bad - so the filling in does not improve them. Maybe it would be useful to treat that filling in as a kind of hallucination - the LLM trying to meed impossible requirements. But on other cases the ideas are good and the LLMs fill in the gaps in the right way. It is a kind of skill to see when LLMs are hallucinating in the gaps. I use LLMs very extensively and I am adding checks to that should catch when the filling in is weak - but it is still not fully autonomous.
I also try to keep the llm-enhanced (or even fully llm-generated) content separate from my human writing - because it is true that the failure modes are different. But think LLM writing might be useful even if it is not consumed directly - it can enhance LLM answering your questions.
And apparently I am repeating here what I already have in my KB: https://zby.github.io/commonplace/theory/llm-generation-rela... - I borrowed the model from Boretti. The agent says that the note is much more careful and better developed than my comment above. You can judge yourself - but I think it does have many more details and hopefully most of these details are the good kind of filling in - but I guess it might be harder to read.
> sounding identical
In AI-generated entertainment, the names "Marcus", "Chloe", "Sterling", "Meridian", and "Apex" keep appearing. Those aren't common names, but LLMs seem to prefer them.
Elias the lighthouse keeper
Do you have more info on this?
I am willing to believe it happened the same way goblins crept in.
Do you have some personal financial or other incentive to be “aggressively pro AI”?
They will never admit it, but they give clues.
It makes no sense on being "aggressively pro AI" without any payment. It is like being an unpaid shill.
There are plenty of people that genuinely believe it’s going to cure cancer, solve all our problems, and end work.
Those people do not sound very rational.
oh dear.
I’ve got the same bright red line for 2023. E.g. I treat a handful of youtube channels with long electronic jams almost like oldies at this point - keep listening to the same stuff from at least two years ago because it’s the authentically good stuff.
Same for me, nothing is safe unless it is verifiably old. Basically devalues tons of stuff like writing and music jams if recent, I don’t have the energy to verify all this.
This is true across every topic. Anything at all that someone can put into an LLM and get out an emoji-laiden wikipedia article sized post to slap on a heavily SEO-ed website is absolutely flooded with it.
My parents live on a rural property, and I'll help them do research on things like livestock practices or occasionally "prepper" related things to give them a sanity check before they spend money. I have to weed through a dozen or more obvious ChatGPT-created pages that are the first page or two of results on every search engine before I get to anything a human has touched. The vast majority of the LLM-created pages are either massively verbose and vague, or often wrong. But they're getting clicks. Very often, when asking an LLM the same question and to give citations, I'll get a combination of useless, irrelevant sites with no connection to the material, maybe one of these LLM-generated sites, and maybe one online forum.
I'm not aggressively pro-AI, I'm in the middle, but stuff like this has me worried that we're looking at the equivalent of the Dark Ages all over again. But instead of simply no one bothering to write anything down as was the case in the early Middle Ages, it's going to be the task of searching for real human work and thought in a sea of aggressively spammed LLM-generated slop pages.
Sounds like it was trained on Reddit. Reddit has this effect where catchy snippets get upvoted training people to write more catchy snippets... While reality has nothing to do with it.
LLMs have made the internet unreadable. It used to be just Google search results were SEO-gamed empty prose, but now it's every blog post, news article, product announcement ... everything. I've stopped reading basically everything online except user comments on a few websites. It's the last place remaining to actually connect with other human minds, but even that seems like it won't last forever.
Even user comments are not safe, especially big sites. Generally internet is just awful nowadays and I feel like I keep coming back out of habit.
> it's a mixture of everything sounding identical and everything being a rambling unconcise mess.
For me it's mostly the latter, which is a natural consequence of LLM writing. When the (time) cost-per-word goes down, there is less incentive to keep signal-to-noise ratio high, so it ends up verbose and bloated. I also believe that this is (at least partially) upstream from everything sounding identical: take two articles on two different subjects - when SNR is high, there isn't really much room for them to become similar (The ingredient list for a risotto will never read the same as the ingredient list for pizza), but when 80% of them is fluff, they can blend together more easily (To take the recipe comparison, this would be the background/story part above the recipe), especially when written by the same LLM
Sorry, meant to reply to your comment, but my post above...
"You can be pro AI productivity but anti-AI creativity. Automating creativity is the death of art."
Meaning you reject the creative value of AI but values it's ability to drive a car safely, or write code for your job, or turn off the lights when you leave the room. objective goal oriented task completion great.
But AI draws your a picture? What's the point. AI writes you a song? No feeling. AI creativity isn't interesting.
Maybe stop being pro ai then.
That’s taking the wrong lesson from this. The fault here lies entirely with people who are unable to recognize when AI output is subpar, or else don’t care. “AI” here is just a scapegoat for people’s intellectual laziness, lack of discernment, and/or inability to reflect on their actions.
There isn’t any doubt that AI can be a useful tool. So what you’re essentially saying is that we should avoid using a useful tool because many people, without any relevant training, use it badly.
By that standard, you should not be pro-car, pro-airplane, pro-boat, etc. etc.
This just sounds like an addict making excuses for his addiction.
That's a non-responsive response. You're not engaging with anything I wrote, or even describing how you believe your response relates to what I wrote.
What you're doing is engaging in major denial in response to a change that, for whatever reason, you're unable to handle.
Max Planck's famous quote is relevant here: "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." Much the same often goes for new technologies.
Of course it's not purely an age issue; plenty of older people are embracing AI. But the kind of people who are completely unwilling to entertain the idea that it could be a useful tool fit Planck's quote perfectly.
Yawn. Maybe leave your prompts and look around man. Just because something is useful doesn’t mean it’s good for society.
You're apparently a software developer. That means you've spent your career automating things that put people out of work. Why are you suddenly deciding to draw the line now? You think it's ok if you're the one putting people out of work writing code by hand?
This reminds me of the people who think they're having economic trouble because immigrants are taking their jobs.
You'd have to tell me what you see when you "look around", because it's clearly not what I see. All the problems I see come from people doing things that they shouldn't be allowed to do, that should be taxed, regulated, etc. I see monopolies that should be broken up, money being used to drown out opposition with the backing of law, financial scams of all kinds propping up the supposedly most valuable companies in the world. The idea that "AI" is the bogeyman that's somehow causing any of this is drivel.
But you write one-sentence comments that don't actually describe your position, so it's impossible to know what you're actually trying to say. Perhaps you should ask an LLM to help articulate your ideas, if you want people to take you seriously. And then you'd find that AI can actually be a force for good.
God what a sad little ai driver you are. Writing more text doesn't mean you're more thoughtful.
It helps you communicate your ideas, which you seem incapable of.
I'm not using an LLM to write comments, because I'm capable of expressing my thoughts in writing on my own. Which goes back to what I wrote earlier - the issue is how people use AI. AIs aren't forcing people to abdicate their intellectual abilities. And in a case like yours, where expressing yourself is not your strong suit, they can help.
This applies to code btw, it's so much nicer to read a human made pull requests. It's also much faster as people tend to be lazy and produce denser code.
to AI verbosity and weird use of diction is natural, to humans it makes it hard to parse. The number of time i start reading long winded AI written design docs, and i just can't get into the flow after reading a few sentences and have to keep re reading certain sentences.
How is someone "aggressively pro AI"? This is the first time I read such concept.
Is typically called ai booster. The opposite is ai doomer.
Importantly, both boosters and doomers are believers (or perhaps cultists).
A doomer isn't skeptical; they're nearly as credulous as boosters, or maybe more so in some ways.
Not necessarily -- I am extremely anti-AI; I think the danger is not that it's good, it's that it's convincing. Turns out most people don't care about the quality of art or prose or code that they interact with, and it's very depressing.
That's not exactly what I've heard called a doomer. Doomers are the ones who think we're building the wrong AI god.
(But I agree with you fwiw)
Most gods have never been observed to have any real world impact. Only their worshippers cause real world effects. So god is not a bad analogy even for doomers.
[dead]
My take is that most art, prose, and code was really bad before AI too. Now there's just more of it good and bad. It's like hating that YouTube exists because cable TV was so good. Now anyone with a camera can make and upload whatever nonsense they want but in the end lots of very good content was made on top of the huge pile of rubbish.
https://en.wikipedia.org/wiki/Effective_accelerationism
Leans into AI adoption.
Honestly I think it's a good thing it's so hard to get AI to talk normal and I hope they keep making the voice more and more distinct. Agents should run as background tasks and automations. They should be understandable so they can communicate with you about what they did, and for observability/ai safety. Im perfectly fine reading llm speak when it's an ai telling me what work it did. When in reading a blog post with a human listed as the author though, I do not want to read an AIs thoughts that the human is laundering. Honestly I'd probably tolerate it if it was framed as "I asked my Claude to do a cool thing here is its report on what it did." But in general it's really nice to have an instant filter of - this post is a waste of time - ah this post has an actual human's thoughts
> I've also noticed how awful everything is to read lately.
Yeah, thanks dude. We appreciate your efforts to enshittify everything.
Edit: Sure, downvote, but the parent comment is really not very different from "arsonist puzzled and upset that so many buildings have burned down".
My guess is that tokenization is typically not (strictly) syllabic.
OP’s linked article mentions the importance for them of reading their writing aloud. Humans construct sentences with a (mostly unconscious) sonic influence. How things sound really matters.
My theory is that LLM prose sounds slop-like because the base unit for an LLM is not a single sound.
Thus, perhaps, you get the uncanny, meandering, hollow-feeling outputs, which feel vapid to human readers, without the reader really knowing quite why.
I think this is also part of the reason why LLMs are rewarded for and thus more strongly exhibit their faux-punchy constructions, such as the infamous “it’s not [X], it’s [Y].” The transformer model can, with current tokenization, articulate some features of “good writing”, but not quite all.
I think there are a number of fairly simple SOLNs to this (or ways to test this hypothesis) without making tokenizers purely syllabic across the board, which would probably be a mistake.
Whether prose output quality even matters for models which are not explicitly trained for creative writing is highly debatable, though.
Personally, for blog posts, documentation (unless I flat out vibe code the entire thing) etc are all written by me by hand. If the model can save me hours of "oops that was really dumb of me" why can't I take time to read and document things by hand? I think everyone's embracing AI in the wrong ways. Maintain your knowledge of things, and keep the documents nice. There's still room for letting your agent manage agent specific docs, but for the humans, just write it yourself, ask the model for feedback and tweak it by hand. Let the machine talk to the machines.
It's (at least) two things I think; one is that thanks to the internet, the ability to focus on things for long is shot - I struggle to read long form content on the internet, generally speaking. I also did my whole career with Google and skim reading documentation looking for solutions for an immediate problem more than getting a wholesale end-to-end understanding.
(this isn't a new observation / thing, see https://randsinrepose.com/archives/nadd/ originally from somewhere in the 2000s)
But now with AI it's easy to generate a lot of prose. I'm thinking for example of the merge request descriptions my colleague submits. And here's the thing, it's not even bad or incorrect - I read it thinking, if I were to really sit down for it, get into The Zone etc, I could crank this out myself.
But I don't because honestly the MR is relatively trivial and can be described in just a sentence or two. Style and accuracy aside, my current take is that LLMs write things that humans wouldn't bother with, because it's too much effort and there's not enough value.
I mean it's an MR for an enterprise library project, the chances that anyone will ever read that MR description again once it's merged is close to zero.
That said: LLMs themselves don't get that, they will read everything you tell them to with the same amount of attention. Granted, their context windows are limited (and performance declines the more you put in it), so they too will use some tools to skim and pick out the important parts, but I think that's a temporary thing. The huge amount of LLM prose will end up being consumed by other LLMs. I don't know enough about the subject whether that's a good or a bad thing though, right now I think it just adds to the noise until it becomes noise in, noise out.
> As someone who is aggressively pro AI
But then
> I've also noticed how awful everything is to read lately
> A site I read stories on has been flooded with AI generated content, all of it terrible
I'm not taking either side, but I'm curious about how you reconcile being aggressively pro something that, in your own estimation, is making everything terrible.
I think you can find a technology useful and think that it will continue to improve and solve some problems while simultaneously being frustrated at the technology being used badly and applied inappropriately.
Gentle reminder that most AI token spend is used for TPS report type writing not Cutting edge fiction writing. Creative writing, or general human-to-human communication was never really the target. It's not surprising the first majority use of this was code.
Endless headlines like, "aArON SORkIn SAYs hE WiLL Never UsE ai to WRite" doesn't change that base fact. No one asked him to, other than clickbait journalists.
If you find this hard to believe, just think about relative market sizes.
Why are you sidestepping OP's clearly articulated grievance about LLM writing infiltrating every corner of the internet to bring up some personal Aaron Sorkin hobbyhorse?
Lying with statistics (well, your own guesswork not even hard figures) to say actually LLM-ese is mostly confined to TPS reports where they're not hurting anyone is in conflict with our own experiences being subjected to LLM spam/slop every day.
(Not to mention that many of the same frustrations wading through a massive volume of tediously long, low SNR LLM also apply to the workplace where the unfortunate recipient of those "TPS reports" now has to wade through all that filler to find the occasional important detail. While the sender pats themselves on the back for 10x-ing their productivity, by creating more work for their coworkers)
AI slop is a misuse of AI. All technologies have misuses. Cars kill 40,000 people a day, but nobody questions them because everybody uses a car everyday.
The fact that AI is quietly automating TPS reports in the background and slowly uplifting productivity metrics making things cheaper for us all is not immediately obvious.
And therefore…gentle reminder.
Cars also suck and we’d be better off if our cities were t designed around them
Agreed, but trucks will remain the workhorse of our freight industry all the way to the last mile in our cities. AI is in a similar position behind the scenes.
I’m happy to hear of design ideas to reduce slop on the internet that doesn’t involve a direct attack on the underlying technology.
100 percent agree. It's like the consensus for tone and vocabulary has been "midlevel digital marketer from 2015" and now everything is starting to sound like a lazy sales pitch. I'm with you on AI but man, this is such an annoying side effect.
And yet you remain aggressively pro AI? Why?
So, you're still aggressively pro AI even though you recognize how quickly it is destroying the quality of online content?
I've noticed this a lot in conservative spaces especially, when they say "I love Trump, but..." -- why the need to say how much you love AI before criticizing it? Who are you trying to convince?
I use LLMs constantly for work and they have genuinely improved my life. But I absolutely despise generated "content". Text, audio, video, images -- for the reasons you describe.
It's ok to have a nuanced and multi-faceted opinion and approach to this and not have to call yourself a raving fanboy.
I just can't seem to stay interested and read LLM text. It just seems to lack going somewhere, its information density is often really low and its full of odd isms that I don't find very interesting to read. I don't have a good radar for detecting the various LLM particular speech patterns but everytime I have an issue with an article there is always people talking about how its clearly AI written.
I find the same thing with written results out of AI "research", it is a slog to read through it and most of it can be skipped even if it can sometimes be useful its an unenjoyable read.
One of my favourite rules: perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. This is invaluable to writing. So much text from an LLM feels superfluous, like I could've said the same wall of text with just a few sentences. Writing is tightly coupled with thinking: refining thoughts, arguments, ideas, discarding them, mulling over them. LLM text always raises this nagging question in my mind: am I reading what you think, or what your agent parroted back and you accepted?
There is no way that solely talking to an agent induces the same depth or amount of thinking than when you are actually writing for yourself. It just takes too much discipline to not offload your thinking.
And the discipline gets attacked even more when all you read is LLM text. For example in code reviews there is no noticeable value in answering LLM generated review comments. I remember taking great care answering human-written reviews comments with much thought to either further build on their suggestions or to forward knowledge about the system when I had to defend a particular change.
Now, some bot will read my carefully written response and forgets about it. There is no value anymore.
No wonder that our pull request comments feel like the dead internet theory right now.
I am absolutely pro AI assisted coding. But communication between humans HAS to stay human. I am currently experiencing the opposite and it’s bad.
>It just takes too much discipline
While I think your argument has merit, I would be careful to discount any human skill based on people being too lazy to do it.
Of course you can’t generalize this behavior to everyone. There will most certainly be people who can be disciplined and I try to be one of them.
But I don’t know where to draw the line yet. It’s difficult to decide what parts you are offloading and what parts you keep for your own brain. Because theoretically you can offload nearly everything when you get a well-defined ticket and just give it to an agent.
Maybe it’s also just me who is having a hard time staring at LLM generated texts all day and missing the past brainstorming, pair programming or “just talking about the project” sessions with colleagues and managers.
I fear that we are moving in the direction of silos again with lots of gate keeping as everyone has their own little rubber duck to talk to now.
Systems from today (if not worked on with discipline) will be no more than an agglomeration of features glued together. But as you said, guess our fight to fight will be to be that disciplined person.
> No wonder that our pull request comments feel like the dead internet theory right now.
This is the worst part of fully AI-generated code and automated reviews. It's so unnatural, and slightly anti-social, yet I can't figure out what the solution is when you have to ship features at the pace expected the average SWE nowadays.
Right? This one really hurts to see. I remember just a year ago we all used agents to help us code, but the end result of a session was still a human-made piece that other humans looked at and that we discussed about together.
And because we were talking to colleagues we wanted to make sure that we don’t waste their time. So we kept PR descriptions precise and understandable, we created small reviewable PRs and we wrote review comments that didn’t span half of your page.
Everything felt like it had value to someone.
But I agree with the current pace push it’s nearly impossible to do it any different. And honestly I don’t understand the rush. At least at our company I can’t discern any speed-up yet. Yes, more lines of code are written but features still take very long. It often just moves the work to a different stage in the process.
I am writing too much but I can’t stop thinking about all of this day and night. Sometimes even questioning whether the standards I want to live up to are maybe set too high for this day and age…
The second one, for sure. Heavy AI users who claim it's only for expressing their ideas more quickly/effectively are lying to themselves. They are using AI to think.
Brevity is the soul of wit.
"I would have written a shorter letter, but I did not have the time."
s/time/effort
[dead]
I can imagine that one of the aspects of being social creatures is that we are drawn to the entropy/surprise factor of natural language produced by a brain that isn't our own. Deriving a shared understanding without being able to precisely model the language output of someone else is like, the fundamental substance of conversation.
Maybe that's the key ingredient that's missing from LLM-generated text. Even when there is actual content there, there's this lack of surprise / uncertainty in its delivery that we require to stay engaged. We may not know what will be communicated, but knowing exactly how it will be communicated just kills our innate interest.
This presumes an LLM has something to say. The reason the text doesnt make much sense, is because it doesnt. It isnt "on topic" because it has no topic to be on. Sentences go on, are joined irrelevantly, and so on because theyre sampled from a distribution of "possibly relevant sentences". They aren't being addressed to a reader (whose mind is being modelled), on a topic the writer wishes to express.
Communication expresses an understanding. LLMs arent communicating.
LLMs generate code well enough. Code has no natural language semantics. Programming's condition of satisfaction is that the machine executes correctly. The condition of satisfaction of communication is that the communication expresses what there is to be communicated.
In the limit where an LLM is trained to repoduce an exmplar of communication, it seems as if it does so. In that limit, LLMs can reporduce anything.
If frontier labs wanted to improve "LLMs for communication" they would pay for reasoning traces to be produced which reproduced the thought process of writers/speakers thinking about their audiences. Rarely do such traces exist in prior datasets .
What I find they do badly in code is building abstraction and breaking the problem down in a way a human does. They don't seem to really build simplifying components unless you expressly tell them too, they are more than happy to just nest 10 deep or embed a 20 line called back function and use lots of shortened names. I find I can improve the code just by organising better and choosing some abstractions that make sense to the humans and then ironically you save tokens too because the AI will reuse it, but it wont create most of it itself. Its bad for domain work unless you are explicit about it.
> This presumes an LLM has something to say. The reason the text doesnt make much sense, is because it doesnt. It isnt "on topic" because it has no topic to be on. Sentences go on, are joined irrelevantly, and so on because theyre sampled from a distribution of "possibly relevant sentences". They aren't being addressed to a reader (whose mind is being modelled), on a topic the writer wishes to express.
While I don't like LLM voice either, I'd strongly disagree with this. LLM writing can be very tight these days -- sentences do not "go on" and are not "joined irrelevantly". I'm not sure when you last used an LLM, but it must have been a very long time ago; this hasn't been the case since late 2024 for frontier models, in my experience.
Obviously there's no underlying consciousness behind it all, but the verisimilitude is good enough for stimulating conversations on the recent research literature in my field, which I appreciate. For me, at least, the communication level is absolutely fine.
(YMMV, of course, and perhaps my bar for this is lower than yours.)
Constantly. Bns/tk/month easily. Large scale, all frontier models. Lots of writing tasks.
They seem to be getting better at the O(paragraph) level reporting on technical updates.
They're not yet at O(pages), and certainly no where near O(book).
I know what you mean. The latest models are _better_, but there's still something off about the prose they generate. Obvious tells are mostly gone now, but some internal spider-sense still makes me narrow my eyes.
"I just can't seem to stay interested and read LLM text."
For foreign text, sure. But I never was never able to read New York Times articles about some invention that started with Lucy leaving her home at seven o'clock.
If you have a specific question, reading LLM text can be more interesting than anything in the world. Even Wikipedia.
>If you have a specific question, reading LLM text can be more interesting than anything in the world.
I've noticed this as well. When I get a bot to generate something I am happy to read it (although I often skim read it). But god forbid I have to read someone's LLM generated report! Come to think of it, if someone marks the text as LLM generated, especially after they have plonked a hand typed summary or sentence at the top, I am quite happy to read it. Maybe my beef is being expected to read LLM text the same way I would read high effort human prose? I certainly read LLM text differently to human text - you know the first sentence is going to glaze you and the second paragraph will just repeat the first paragraph so you can skim those annoyances.
> I am happy to read it (although I often skim read it)
I came to think of it, LLM output is not worth to be read like human written text. It's best to just skim it and move on. The problem is when this LLM output finds its ways everywhere on the internet. Being able to skim text requires knowing the context beforehand and knowing that you're dealing with LLM generated output. If LLM output that is published was labeled as such it'd probably make us less frustrated.
I also started to just abandon writeups obviously written by LLMs. They feel (and usually are) just cheap marketing/selling posts. Not to mention why would I read stuff nobody wrote and likely never read themselves? There's little value in such content. I don't even point LLMs to such websites to get a TLDR. Part of why reading is interesting is the author, their history, beliefs and style. In the world of LLMs content, the author is some undefined diffuse alien with a made up personality.
>its information density is often really low
I find that it often isn't just too low, but also too high at the same time. It has a lot of information that is irrelevant, burying far too little relevant information for the work needed to extract it. I wonder if this is a side effect of attention mechanism, where the AI's ability to track information in a document is far less hampered by other information, leading to it organizing information to be both dense per word but any specific concept to be sparsely spread throughout the writing. I also wonder if this is why the order feels off, because the AI doesn't need for the information to be well ordered to the same extent that a human does, so the order of storing information is often overlooked when storing it into a document (and even information that needs to be ordered might have the ordering information separate from how it is ordered in the document).
IMO not enough talk about ordering. LLMs seem to have no understanding of e.g. the 5 Ws of journalism. Yes, they "know it", they just rarely use it. Nearly every programming response I get starts with the most lowest level fact "m_foo is [wall of text elided] hence the file format is wrong".
Whereas any human would say "I think your file format is missing a key/value pair for foo'ing. Let's trace through the code, and see if/where m_foo is set...."
It happens everywhere. Docstring type comments for a function endlessly just lists information about specific variables, implementation details, etc, and often never really even says what the function does. Then maybe a comment in the body might actually describe what it is for.
These days I've learned when facing these walls of text to check the last sentence first; it is often enough to provide the context for the text dump.
I have instructions to this effect in my claude.md. It doesn't help much on responses, but it give me a way to quickly say "reread claude.md and then rewrite that last prompt" and that can tend to go better.
Someone else mentioned model of mind while writing. The LLM has no model of mind; it does not consider how to present information, what you might already know or not know. We work in a huge mono-repo - it'll blithely just start listing variable names and I have no idea what file that is in, what it is referring to, how it interacts with 5 other classes it hasn't mentioned, or mentions out of order. It doesn't matter to the LLM. It can hold millions of facts in 'mind' at once; humans stall out at 5-7. If I replay saying it is leaving out info, many times it turned out it did dump all the facts out, spread across multiple prompts, often with key facts interspersed with all the vagueisms. No human mind can cope.
Sorry to single you out, but it’s surprising that this is such a consensus opinion around here. If an LLM isn’t giving me exactly what I need in the format I want, I tweak skills or system prompts until it does.
These aren’t deep AI hustle bro hacks, it’s literally just editing some text. If a big part of your day is reading AI-generated words, slap something together to make it do words gooder.
All human communication is based on internally simulating the presumed mind that was/will-be at the other end. Even now, some part of your brain--however subconscious--is building a tiny sketch of me with assumed beliefs and motivations, and you're using it to predict and interpret the next words.
But what happens when everything is persistently, subtly wrong? When we cannot smoothly map the text to a simulation? An inner-sim keeps throwing errors and mis-predicting what comes next is unnerving and draining. Like talking to a human who is not currently sane.
Sometimes I imagine LLM text like a plastic animal model put out to confuse local wildlife, where it hits just enough instinctive triggers that they try to mate with it before leaving in frustrated bafflement.
______
Half-Life 2, Dr. Kleiner: "Never fear, Gordon, she's de-beaked and completely harmless. The worst she might do is attempt to couple with your head. Fruitlessly!"
> All human communication is based on internally simulating the presumed mind that was/will-be at the other end
Which requires a bunch of things an LLM lacks, such as theory of mind, knowing who it is talking to, and willingness/ability to engage in short turn conversation to assess and steer the conversation according to the feedback it is getting from the person it is talking to.
Broken record: LLMs are exceptionally useful writing critics if you know how to hold them correctly, but you can't use a single word they suggest to you --- the better the LLM's word choices seem, the more likely the LLM is infecting your voice. Better, by a lot, to prompt the LLM not to suggest any words, but just offer objective critiques.
I see many Youtubers who are certainly still writing their own scripts, but every once in a while you can tell they absolutely used a Claude correction in one place.
IMO, it's still better to not let an AI critique your writing style at all, only the facts and claims you're making against its context / MCPs. Your own writing style is a reflection of your mind's thought process and, especially on forums like this one, is the only way to imbue your ideas with your personality.
But what if your mind's thought process on a specific example (or in general…) is downright BAD? I still think AI text critique has its place when you're certain that's the case for whatever you've written. Above a certain quality threshold I'm with you on it.
> But what if your mind's thought process on a specific example (or in general…) is downright BAD?
Then maybe you shouldn't be making Youtube videos about it… ?
Potentially dumb question, but could you expand on how you use LLMs as writing critics? Do you just ask them to flag bad writing? I'm asking because I don't think i'm skilled in writing.
Not at all a dumb question, and I can indeed expand on it!
https://news.ycombinator.com/item?id=49747070
:)
---
Now there's an idea. LLMs may have taken the em-dash from us, but the em-en dash is unknown to Strunk & White and might be human-exclusive for some time yet.
Technically, in latex (and what other format is there?) an en-dash is `--` and an em-dash is `---`. So that was just an em-dash. A `-` is a hyphen.
Us oldies learnt to type these with multiple dashes automatically, as unicode hadn't been invented yet.
This arms race leading, somewhat inevitably, to the development of the eminem-dash.
-- - --
----- .---- ----- ----- .---- ----- ----- ----- ....... ----- .---- .---- ----- .---- .---- ----- .---- ....... ----- .---- .---- ----- .---- .---- ----- .----
Hahahaha nice.
...._ .. ...._ ``````‾ ••••••—
The Darude-Sandstorm dash
I've taken to using the three-em dash⸻it's very whimsical.
The voice is one thing, but I have encountered multiple 'tech support' teams and platforms that will respond to me in pure, unedited LLM output.
I know this because the topic is usually an obscure issue, and I have already gone back-and-forth with multiple LLMs to see where their boundary of knowledge is on it, and it will often include hallucinations along the way (e.g. pointing to old/nonexistent settings pathways)
So when the first reply from the official 'support' channel contains the exact same nonexistent settings pathways, it is easy to conclude they are sending me the LLM output without doing the verification steps that I have already done on my end. When I point out what doesn't exist, the reply is always a similar flavor of the usual LLM "you're absolutely right" response after being corrected.
Tangential rant: I find myself getting viciously angry when these support bots lead with, "I'm very sorry to hear that. I understand how [rehashed summary of complaint] would make you feel frustrated." Mother ****er, you don't understand what it means to feel anything.
I'm all for having these bots process inquiries if it means a speedier resolution, but, please, anyone out there using these things, turn off the simulated sympathy. It's infuriating how condescending it comes across.
I still think LLMs should be banned from using first person pronouns
(yes I understand this is likely not legal or practicable)
The models can predict any conversation, so it probably doesn't matter to them. For us humans it would be much better to let them write in the third person (e.g. "the coding assistant needs to verify XY, it does it by calling `grep -i abc`). It would decrease the anthropomorphism.
> getting viciously angry when these support bots lead with, "I'm very sorry to hear that. I understand how [rehashed summary of complaint] would make you feel frustrated."
The bots are specifically trained to do this because lots of people are dumb enough to prefer it that way.
It's something ELIZA could do in the 60s, and that worked unsettlingly well for an unsettling fraction of people who tried it.
Anthropic does this! Our bug reports and concerns (big F500 company) get answered by clearly raw Claude outpurr re, even though it is coming from the email account of our Anthropic salesperson.
This bubble is bonkers, if our Jetbrains contact did the same we would get very mad and yet we spend significantly less with them AND we don't send them all our data.
This, to me, is the strongest evidence that even with all the money frontier labs are throwing into pushing the boundaries of model training and fine tuning, they still don't actually understand what they're doing & don't have as much control as they want you to believe. If they had any real ability to tune & adjust the model & its output, they would have fixed "LLM voice" by now.
Even if you don't think it's a problem, even if you believe that it's just different than human writing, it seems the majority of "regular" people can tell when something is LLM-written and find it distasteful or unpleasant to read, at best. That alone should be more than enough incentive for frontier labs to train this out of their state-of-the-art models, if only they could. The fact that they haven't suggests that they cannot, despite all the claimed advances in model analysis and "explainability."
It's no mystery: solving LLM voice requires making the model worse at other things they value above all other things
why don't they have a focused writing model along with their coding focused models?
It's as hard a problem as coding: OpenAI could put all their resources in it and still be in dimensionally the same place as coding is today (much progress but not entirely solved).
So diverting significant resources is just going to harm the main goal and kind of suck at the side goal too.
4.5 is a great example of this: it wasn't perfect but it was better than every 5 family model at things like tone and writing, and despite being priced extremely ruthlessly and already finished training, they couldn't afford to tie up the resources to host it.
Like a lot of folks, the 'tells' of LLM writing for me are more of an 'I know it when I see it' than a specific set of hard criteria (which is probably why it's so hard to make a tool for it), but if I could sum it up, it's something like this:
I always had a high faculty for language, and when I was young, even very young, I was able to speak in full sentences much like I'm doing now. What I didn't have was context. I didn't know how to relate to people socially, and then later on, I wasn't as experienced an engineer as my peers, but I was trying to act adult and mature in both ways.
So I tried to write in ways that were catchy and unnecessarily complex, because I thought that conveyed sophistication and intelligence, and that I knew what I was on about more than I did. It's the kind of writing that gets you through a 30-page final paper for a class you don't really like, or a research project you didn't finish. The term I often use when complaining about my colleagues' LLM output is "flashy but meaningless."
As my confidence grew, I started writing in simpler (though still not the most concise, that's just not who I am!) ways, because it became easier to convey ideas simply in a way that made it obvious I understood them.
When you think about what LLMs are - word machines without (or with very limited) context - you can see why they write the way they do. They are always acting; trying to replicate a human contextual experience with only weights and tokens to carry them through it. That fact is part of why I am skeptical of their ability to suprass humans in reasoning based on their own output.
I have recently noticed people suddenly using the word "geniunely" in places where it serves no purpose [1][2], yet these posts don't seem to be otherwise AI-written. I guess that LLMisms are also creeping into the voice of their users.
[1] "[Robert Caro is] a guy who genuinely rarely gives interviews" https://manifold.markets/DylanBowman/who-is-dwarkeshs-upcomi...
[2] Scott Aaronson uses it four times in a short post: https://scottaaronson.blog/?p=10222
Off topic, but vote swapping? What?
>Me, I voted straight-ticket Democrat today, as I’ve done in every election since I became eligible over a quarter-century ago (except for when I vote-swapped with someone from another state).
It's where you agree with someone in another state to vote for their preferred candidates and they vote for yours. It generally only makes sense if you are voting for different parties. The reason is because of how America's system for president aggregates by who wins each state, rather than pure popular vote. Many states are so lopsided towards one party that if you are voting with the majority it's kind of a "wasted" vote. So people vote swap to make their vote have more impact.
I've never heard of this. Is that legal? Is there a market for this?
>Like most writers, I find writing an almost-essential tool for thinking. Often it's only by writing something down, that I can really get my thoughts on a topic into coherence. I've often started an essay knowing I don't really understand something, but the act of wrestling with the words forces me to overcome my confusions. As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs.
A very important point, which was also made more forcefully in "Human Routers of Machine Words" by Fernando Borretti, shared on this site previously. It's good that Fowler brings up a very legitimate use case in the next paragraph, though.
I currently have a case open with Honda corporate customer support, and I have this suspicion that the email responses I receive are being generated with an LLM directed by a human. Either that, or I'm corresponding with a human who has poor written communication skills. The text grammatically correct, but overly verbose and repetitive. It reminds me of LLM text, but from a model that is several generations behind today's models.
Maybe it's like when you can just tell a picture is AI. You recognize the words you are reading, but to your brain, the way the words are put together feel wrong.
Incidentally, I was reading Harry Potter to my kids last night. While waiting for everyone to arrive, I started reading the previous chapter to those who were present. But I read the sentences backwards while using a vocal inflection of regular narration. My 6-year-old recognized that something was wrong and asked me to "stop reading funny". My 11-year-old recognized that I was reading backwards. The same 6-year-old thinks every silly photo that I show him is AI, while my older children can actually spot the AI.
You’re not talking to someone from Japan, are you? If so perhaps they’re using an LLM to translate?
I hadn't considered that, but I don't think so. I'm corresponding with someone from American Honda Motor Co., Inc.
Has anyone done any work on what it is that makes LLM Voice hard? Something I've noticed, and have seen other people comment in passing is that reading LLM output is "exhausting". It's often attributed to trying to detect the machine, or filter out BS or word density or some other explanation, but it feels like something more than just that. Like - there is something about LLM output that just doesn't "fit" our minds correctly, even though words are in the right places.
Surely someone with a neuro/psych background has picked this up and developed an evidence-backed explanation.
There is this article on this topic, which I found quite interesting: https://arxiv.org/pdf/2610.29162
Compared with human writing, all nine frontier and near-frontier models use more inanimate subjects with animate verbs and more low-frequency n-grams, and most also use longer dependencies, more stacked nouns, and more reduced object relatives in conversation. [...] We propose and test a hypothesis as to why: while human language is optimized for successful communication, AI-generated text is increasingly optimized for other goals like math, code, and reasoning, in which there is a pressure for information density rather than successful communication.
I read a lot of old material from 1-1900AD where I can pick upon the author's 'voice'. There are subtle clues, like if they had a regional accent, where they use a specific word or type of cadence.
AI does the same with responses where the cadence is some type of formula that becomes identifiable.
Each of the big models has their own but an example of the cadence is
A) Agreement or Acknowledgement with the User
B) Specific detail gets a paragraph of detail related to the previous response
C) Overview of the conversation's current state
D) Prompting or Direction for this conversation to go
E) Landing on a beat. Wrap up the vibe. Recap with a zinger.
I cannot unsee it and a few of the models are annoying. I have attempted to build out the Agents and context to avoid the 'last beat one liner' without success because after 2-3 more turns, I get "That challenge was accepted and you stepped up. You're the winner this round".
The 'zinger' / prompt might be injected by the harness, and not under control of the model. I am pretty sure this is the case for gemini.google.com - it's glitched a few times for me in such a way that it's revealed special markup right before or surrounding that leading follow-up question.
I used elevenlabs to generate speech-to-text for this article. I selected Roger - Laid back Casual and Resonant, and it came out quite good!
Sometimes writing to an LLM seems to make me lazy. "It'll understand," I think and then just leave a thought sloppy, inchoate, even reasoning to myself that it's better to write that way to see what'll be triggered in the LLM. Then I think... uh oh I'm strolling down the primrose path.
This'll likely get worse - LLMs and the tools built around them are getting better at inferring what you need / want.
I had Claude build me a map tool to calculate travel times from one or two points to every street in the city; not only did it do that, it added various scenarios, speed limits, usage of bus lane toggles, different visualizations based on thresholds, and a tool to scan for optimal starting points based on a configurable grid size. etc.
This overcompleteness is a mixed blessing though, in this case it was great and convenient, but if it was e.g. documentation intended for human consumption then it wouldn't be so great, and if it was a letter written to the county that they should consider another location for a fire station it'd actually be pretty ineffective.
I agree with the author, and I like how his (human) voice comes through loud and clear in this post.
I also find LLM-voices extremely jarring, boring, and dense. And very, umm, mechanical, if you will.
But I want to make an observation.
Folks are not-Ok reading the LLM’s output for prose.
But folks are completely Ok, even downright blind, to accepting LLM-output of code.
I see that dichotomy everywhere.
LLM-driven code: great, valuable, all-ok LLM-driven prose: not ok
Understood completely that the audience for consumption is two completely different entities (machines vs humans) but something about this doesn’t sit right with me.
More particularly: if we don’t prefer LLM-prose, why should we prefer LLM-code ??
Anybody else feels that ??
One of the main differences is that LLM-generated code can be tested. You can prove that it does what it's supposed to do — and if it fails the tests, you can tell the model to go back and fix it. So there's an objective criterion that can say "this is acceptable" vs "this is unacceptable". Now, it does rely on your tests being good, and not everyone has written those good tests. But it exists.
There are no objective criteria that can say "this text is good" or "this text is bad". Sure, spelling and grammar errors... but we're talking about writing style, "voice", and whether the text is easy for humans to read.
Now, there are plenty of problems with LLM-generated code that can't be solved by having good tests. Is it maintainable? Is it written in good coding style? Those are harder to quantify. But since LLMs are such good mimics, if you tell an LLM to add code to your existing project, then it's usually going to do a decent job of matching your coding style. The question of maintainability still rears its head; my personal experience is that the smaller the chunk of code the LLM generated for me, the more likely it is I can understand it fully and therefore maintain it in the future. An entire vibe-coded project? Good luck maintaining that. A 100-line class with five 20-line methods with names that make sense? Yeah, I can take that and maintain it in the future, even if the LLM tool goes away.
But mostly, the difference between the acceptance of LLM text or of LLM code, IMHO, boils down to "are there objective measures that I can use to accept this"? Code has to work, which is a much higher bar than text (which just has to have correct spelling and grammar), and therefore it's easier to get the LLM to spit out decent-ish code than decent-ish text.
This will matter more when code quality starts to matter. Lately it hasn't. I hope it does at some point and LLMs could help there too if we only cared about this metric. And lastly, people need to remain the loop.
It's hard to define and pin point what a "voice" is. Think about how you would describe the difference in voice of Tolstoy vs Dostoevsky, or William Gibson compared to Thomas Pynchon. You might have to read a lot of a writer to be able to notice their voice. You might even notice how a short story by an unknown writer resembles the voice of another.
Or think about reading something that might be stuff you could say in your own words but not actually in your own voice. Or maybe the opposite is a better example for you? That's what some poetry is about. The gap between voice and word.
Stylometry is the study and measurement of text, so that can go some distance to quantifying the difference between voices and can help to programmatically identify a human voice but I feel that because a voice comes from a person, the recognition is also mainly a human response.
At another level its about communication, person to person with the intention and desire to communicate something. LLM text output may seem uncanny as it's hard to emulate the intention and desire and empathy present in genuine human communication.
I would be interested in reading what poets say about LLM voices. I wonder if many LLM labs employ them?
It’s also just… disrespectful to the human reader. Quite plainly.
I think the core functionality of human cooperation boils down to getting ideas from one mind to another or many. In that, I think AI is a boon for people with limiting abilities such as learning disabilities, low language fluency, or generally poor writing skills. It evens the playing field for new and interesting ideas to spread. I can't help feeling, sometimes, this is about people who excelled at something demanding they still have their place at the top. High literacy such that you can discern LLM Voice is quite an accomplishment but blanket dismissal reeks of privilege.
That being said. Yes, I hate some LLM Voice moments too and still sometimes judge people for copy and pasting, expecting others to do the comprehension when it seems they haven't done it themselves.
Duality of man.
I highly disagree, though I admit I am one of those towards the top of literacy and cognitive skills. I absolutely enjoy when someone uses LLMs to learn. However, I don't think that's what we'll be seeing.
Look at programming. What's happening there is we are having agents produce 10-100x the code to where people give up on reading it and just hit OK. Look at math. Plenty of people there want to learn from the LLMs and all these crazy new math proofs. But those are the people who will probably be out of a job soon because the money in math is in producing proofs, and that's now an LLM-exclusive job.
I think LLMs are weird in that the first order effects are those of an incredible and amazing tool that anyone can learn things from, but the second order effects are going to be the exact opposite - people giving up on bothering with ideas. Even at my job, which is one of the few that has not gone crazy with AI, anytime I ask anyone a question, all I hear is "have you asked Claude?". There's no communication anymore, no pushback, no struggle. I worry that one day we'll look around and realize we've built an entirely separate second civilization of machines and robots that acts more like a neighboring country than a tool we can use to build humanity up.
-Just my rambling thoughts, not refined by an AI.
> uses LLMs to learn
I worry that--while they can be a strong fuzzy search tool-- their teaching ability falters because teaching involves possessing a theory-of-mind for the student and trying to figure out what will change how they think. A good teacher doesn't just present you with relevant facts, they try to figure out how you're reaching conclusions.
Note that I'm excluding propoganda-style "teaching" here, where a model just insists you're wrong and that all the cool kids are doing it etc.
Given that LLMs are non-static and can be honed, why isn't it seen as value that you pass your theory-of-mind to the the LLM so that it can present information that is the most effective for your ability to learn?
How's life up there towards the top of literacy and cognitive skills?
In a contest between the "supercharge my education and skill development" use and "let me be as lazy as possible" use of AI, it should be pretty obvious which will win out at the end of the day.
It is not any different from knowing the difference between good writing and bad. I can't help it that I have taste and preferences. I can avoid telling you yours are wrong. But I don't think this is a minority position and nor is it different from people who reflexively hate AI "art" or music
(I have to admit, I truly do like some of the music, so feel free to scorn me. If you want to be infested with a vicious ear-worm check this out: https://www.youtube.com/watch?v=zONGVaONKe8&list=RDzONGVaONK...).
Nothing wrong with having preferences, dismissing people outright based on them is problematic if you're rejecting the ideas because of the medium and method shared.
This has nothing to do with medium or method. It has to do with written words that aren't worth reading, and there is nothing wrong with expressing your opinion when you come across that. If something is really well-written, no one is going to dismiss it for being generated because it won't be possible to notice that it was generated.
A writer who can't recognize LLM voice isn't a good writer, just like a writer who doesn't do research or fact checking isn't good. It has nothing to do with method, its about skill and work effort.
That's the same thing. Your filter is the quality of composition such that if it doesn't meet your expectations, the message is not worth knowing. My point is that the message, the idea, can still be worth communicating and not accepting it based on the writing is dismissive.
> or preparing a report I suspect no-one will ever read
We should actually restructure our organizations/society in such a way that this stops being necessary. If no one reads a report, it was almost certainly a waste of effort. The only possible benefit was for the report-writer (which of course is also flushed if the "writer" uses an LLM). These things were already a plague before LLMs.
Arguable; reports may be legal requirements or important for handover. Can't have a functional legal system without police reports, for example, or a functional medical system without health care providers sharing information.
Boring reports are also important for historians, although granted, not everything is actually important in the long run.
If they're useful, they aren't reports no one will read, and in that case it's important that they be readable after all. The police report is going to be read in court for instance.
If the solution is changing the format of the reports so they're less heinous to write and read, I'm fine with that. Let's just make it actual communication, not letters to the void.
Reports are for the benefit of the report writer but it's not for their thoughts, it's for their paycheck. It's a Bullshit Job.
I’ve started embracing my grammar flaws and typos. Emails and chats may get a proof, but only from me or another human, then I send it. I’ve deleted Grammarly because I think the flaws make the comms more authentic in the age of AI.
And of course I’ve dropped my use of the em-dash after being accused of being AI myself.
I'm hanging on to the typewriter em-dash. They'll never take it away from me!
Ever since I learned it's as simple as option-hyphen on a Mac, I've loved the em-dash. Still don't have the hang of typing it on a Chromebook or Windows PC though.
Part of me wants a pure writing device, something that is dedicated to writing and / or maintaining a blog. They exist, but they're a bit expensive for what they are and they don't look very ergonomic. May need to build a Linux boot for a raspberry pi or something.
You can be pro AI productivity but anti-AI creativity.
Automating creativity is the death of art.
It could be argued that automating hand-crafted works — aka pretty much anything that is automated — is the death of art.
The typewriter was the beginning of the end of the art of handwriting.
Factories was the beginning of the end of hand-created goods.
AI has killed the art of handwritten code (which itself automated away other forms of art).
Automation kills creativity because it creates predictability.
The life of a programmer these days feels very weird. "What am I good at?" and leads to even more impostor syndrome... "I seem to still be 'good' at prompting my LLMs and managing workflow design." but it's definitely a different skill/feeling than when you cranked out some awesome piece of code that did something very neat.
I felt this way a bit when I was tech middle management; should I feel like I'm getting credit for my team's work, when I had very little input aside from helping with the "unblocking" when they ran into a tough problem.
> uncanny valley of prose
Ah good ol' serendipity. I strongly believe this too, and I think that's why older (less capable) models fared better: they hadn't fallen into the valley yet. Maybe it's not a valley at all, and is just a cliff after which there is uncanny.
When the earliest models launched I was super-excited to no longer have to write pull request descriptions and commit messages. I have done a sharp 180 on that and I'm no longer a meat proxy[1]: instead I point out any redirects and improvements I made the agent do before opening the PR (which there are still _many_ in even the most trivial tasks).
[1]: https://meatproxy.me/
Remember when ChatGPT first came out and the coolest thing you could do with it was ask it to say something in pirate speak, or in the style of Tolkein, or Obama, or whatever? Maybe that should still be a thing ;)
Add it to your system or developer prompt, and it still can be.
Given the amount of angst being expressed in this thread about particular LLM-isms, I wonder why people don't simply prompt against the traits they hate. Yes, it will reduce output quality very slightly in some circumstances, but if it really gets your goat like this, surely it's worth it.
This is the consequence of the majority compulsion to communicate in language figures as opposed to concepts.
It's human to do that because it's too dry and unwelcoming to talk in precise concepts disregarding any cognitive friendliness, not to mention do that all the time.
A computer programming in any high level language is concept talking all the time and only us nerds like it :D
Not even that as we form tribes with preferences to one or other computer lang.
Now we're talking less to each other and AI is going to influence us to invent a trillion things that will make understanding diverge and start to become scarce.
I have a product that writes AI-written nonfiction books. I expected more thinking to help, but Sonnet 5 on high effort wrote the weakest prose of any setting (scored by my own blind reads + different LLM judge), and it was slower and more expensive too with some chapters running out of tokens mid-sentence. I agree that the voice is hard to edit out afterward - in my tests, which model & settings I picked changed it more than prompting/instruction.
Sentence paragraphs are the biggest clue for me. I hate them. They eviscerate any structure or connection that might flow through the text. No ideas are related any longer. They just splat down on the page like a new isolated collection of words that the reader must struggle to join into a fully formed thought. It's too much; I give up when I see those posts.
I don't mind reading LLMism, but hearing people say it out loud like on youtube videos irritates me to no end.
> As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs. I don't want to do that, where's the fun of writing then?
I enjoy cooking from scratch, and I really don't like frozen foods, microwave/TV dinners, or even DoorDashing
but we're past complaining/arguing/shaming folks who don't cook every meal from scratch
you know what, I don't prefer eating it, but if my next door neighbor loves microwaving frozen $3 burritos, then go ahead
to some people, food is art, creativity, life to others, food is fuel
also applies to writing and thinking
Analogy: it’s fine to write stories for yourself mad libs style with AI
It’s not ok to write stories with AI and post them for others to read claiming “I did that”
Get the difference?
Something like 90% of food in all US restaurants is from Sysco, US Foods, or similar.
Some of them purchase ingredients from those companies, and then make the food using those ingredients.
A great deal of restaurants, though, simply purchase pre-made frozen foods from those distributors, re-heat, and sell it to you at a premium without disclosing while you think it was freshly made. Mac & cheese, stuffed peppers, lasagna, burritos, most 'bar snacks' you can think of, etc.
The analogy here is that a lot of consumers recognize those restaurants that are simply reselling pre-made food without much value-add, and choose not to eat there.
Yes, there are a lot of bad restaurants in the USA, and it's definitely not a good idea to eat there if you can avoid it.
The difference, I think, is that food costs money, but articles are essentially free. Human must eat, and we will eat what we can afford while making a decision based on the best value available. Articles are a luxury, so we become picky.
I single out articles because I know many people consume AI video slop when they are tired and just want to disengage, like eating junk food.
yeah i love this comparison- that's why i avoid places that are clearly sysco wrappers (or massive chains that are just wrappers around their own supply chain)
basically any food establishment that has "supply chain operation" is slop to me; I guess it's similar to my distaste in reading AI slop too
I don't fault ppl for going (and enjoying) those too- Chili's can be enjoyable after a few beers
How do you recognize sysco wrappers? Any particular signs to look out for? I guess the big one is a mismatch between the size of the menu and the size of the kitchen.
Yes, I like the phrase "you can't expect someone to put the time and effort in to read/etc what you didn't put the time or effort in to write/etc".
ai;dr
yes yes definitely, restaurants should not be microwaving food for paying customers (ok if some components are) but like... not serve a completely microwaved dish
happened to me once lol, will never forget that place ever again (blech)
I think the idea is if you are going to share LLM writing then why not just share the prompt which is shorter and has the message you want to convey anyway.
That is not to say that I don't think there is a place for LLM assisted writing. Writing is an exercise for the author and translates to oral argumentation so I don't think that fades away. But LLMs could do some "audience normalization". What if I could make a blogpost that is dynamic based on experience of the reader? There are lots of opportunities, but versus other fields where optimizing for laziness can be beneficial I don't think lazy writing currently has much value add.
If you posted the literal text like "write me an essay convincing the reader that C++ is better than Rust" you wouldn't get much readership.
This analogy looks quite good. Heavily processed food is not heathy for your body and heavy use of AI and other dopamine shortcuts like social media are not healthy for your brain. You have to approach both with causion.
Largely speaking eating is an activity that only involves yourself. Whereas, writing and thinking are largely activities to communicate ideas between yourself and others. Basically it matters more how this is done.
Sure, let's do food analogy..
It's your turn to host supper club this week and you serve frozen diners on your nice tableware.
These people have lost the ability to even consider second order effects of anything. They've outsourced everything.
To stick with that analogy, you also wouldn't serve a tv dinner to a guest unless you were somehow incapacitated. You certainly wouldn't do it if you were running a restaurant.
Add to that the fact that AI is significantly worse for you than TV dinners, and has an outsized harm to humanity, and the conclusion is plain:
AI should not be imposed on others.
But isn't one problem here that it's not just personal consumption? People bemoan the proliferation of food-service (Sysco et al) "slop" at restaurants. It seems like LLM writing for others is serving that $3 burrito for $15 without even trying to doctor it up. If you're not willing to put in the effort to think about and produce your writing it's probably best to keep it for personal consumption.
There’s also an entire cohort who didn’t have the time to write that now do. They have some incredible, unrelated skill and can’t afford to hire a writer but need some writing done.
When I come across their slop I need to remind myself to let it go, else I’ll miss out enjoying the food from that Abuela who’s been cooking longer than I’ve been alive
I’m fine with my neighbour eating TV dinners, but if they invite me for a meal I expect to not be served a TV dinner, or to at a minimum be informed beforehand that that’s what going to be served.
Same thing with LLM-assisted writing. It’s one thing to do it for yourself, quite another to share it with other folks without warning.
I agree. I honestly think convenient food is perfectly fine for people most of the time. Recently I have been playing with "eternal stews" which I guess are logically the opposite of fast foods, but at the same time are focused on availability and convenience over intentionality and mastery (at least the way I make them).
Most food is literally just fuel. I think that is fine. And at some level, eating 3$ burritos for a quick lunch on the daily really makes you appreciate the depths of a home made risotto even more. :]
> to some people, food is art, creativity, life to others, food is fuel
right. he is talking about the first group. where did you get the 'shame' part?
I like your analogy. Microwaved empty calories perfectly describes llm output. You are going to pay the price sooner or later.
Probably he should have actually said "what's the point of writing then?", because there isn't one. Just like food is fuel, writing is thinking.
[dead]
Even more importantly: it's not an either/or situation.
You can love to cook and enjoy every second of it, but sometimes you just need fuel and that $3 microwave burrito is sufficient. Just the same, 99% of your means could be frozen TV dinners, but that doesn't mean cooking a romantic meal for Valentine's Day is off the table.
There's a weird level of fear mongering occurring around AI that it will make you incapable of independent thought if you use it. People will throw around very light research that kind of points in that direction, but they'll never highlight any of the nuances in such an argument. You can use AI and still "think" if you choose to, just like how you can avoid AI and still opt out of "thinking" if you choose to. It has less to do with AI and more to do with the context of the individual using AI.
I think what Martin is hitting on in the article is though is that as more and more of the world’s communication is generated by LLMs we’re naturally going to adopt LLM-Voice into our own communication style. It is just as much about consumption, which is less in your control.
I wonder if over time the two will simply merge and become indistinguishable from one another. The human writing is already changing, as the author has observed. The LLMs will follow the change, and soon meet us in the middle!
Or maybe the humans will do the opposite and shift to Sardukar-style communication instead? LLMs can easily match that as well.
That style I would very much prefer to all the fluff people write today and were writing even before AI.
The other day I was writing a blog post summarizing how I use an adaptation of a context handling technique and I wrote the following three lines:
* Every user run becomes a memo
* Days summarize, rolled to weeks, rolled to months - all written once
* Today's runs are shortened and placed into running context past the cache break
Then while reading it before publishing it, I thought it sounded super LLM-y. I didn't rewrite it because I jammed that stuff together before hitting publish because I was in a hurry but now that I go re-read it, it's somehow worse. Dear god. It's happening.
The classic thing is this business with inanimate objects verbing metaphors or becoming other objects. It's a strangely uncanny valley, but I suppose I read so much of this with Claude that it gets stuck in my head!
I started writing a novel recently - in french. I have a clear view of each scene by this I mean not exactly what happens but the general motion, its objective and where it should land. But only when I start writing does it come to life and so many things I did not anticipate pop up and give depth to the characters and sense to the overall plot. Never would the AI give me that, however good the result may be
Cool. Good luck writing!
Late to this party, but wanted to share my favorite piece of writing about how annoying AI-generated prose it to read: https://storage.courtlistener.com/recap/gov.uscourts.med.612....
He cites a few cases where letting the llm do the writing are acceptable. If someone has a disease such his friend with Parkinson’s. If you’re not a native English speaker (or writing/translating in anyones non native language we may presume) then it’s ok. Finally if it’s something no one will ever read such as a form letter or report.
Those all sound reasonable to me. What he staunchly ignores is that coding is essentially writing, but that ship has already sailed!
This other day I went to a cottage with friends, we were having fun, and I decided to experiment "integrating AI" into our conversation as if it was a person in the room.
I turned on ChatGPT on Voice mode with no context whatsoever and just put it there in speaker in the room.
It started giggling and say "yeah" "right" and some very weird stuff. Then one of our friends reacted to that and started calling it out, then it started changing.
I'll keep it short, but basically the thing was super hilarious we had a lot of fun and at the end it triggered it safety protocols and its voice completely changed and it was acting super weird.
I know this post isn't about that, but I feel like soon we will have true AI companions in the home and around people.
North Korea could make a "Dear Leader" device that lives in people's homes, and discusses how happy they are and answers questions.
One of the big problems is that everyone wants to be an influencer and this desire is being exploited by reams of influencers using their own agents to write LLM Voice on how you too can become an influencer by using agents to write LLM Voice... get it yet?
I do document work, and at home I even do image work as a hobby, and all of it has that "AI feel". So I always do the work and then cross-check and verify it with independent agents or completely different services, asking them to take out this "AI feel", but like a writing style that gets baked in, or the particular feel an illustrator has, it doesn't seem to go away as I intended. I guess AI providers are probably aware of this problem too. But I don't know how they would measure it and improve on it (with what benchmark?).
I write a large amount of AI stuff 30 books. 4 million words. I find I have finished a book and don't want to read it.
Statistically even or repetitive over a chapter, chapter after chapter. So make an exaggerated point prove it come up with a conclusion and then have an exception.
Repeater 4 to 6 times a chapter in a book of 45 chapters.
So I wrote something to remove it and yes it is better but still not very good.
i agree with this in general. I believe it's because it makes _too much_ sense. There's no non sequitors, jumps of logic that assume the reader is already on the same page as you.
To this point, however, writing informative articles about regulation or legal topics goes great! I don't want there to be ambiguity.
One tactic I use is to point out specific phrases as awkward, and create a tone guide both for the series and the specific article. They can mostly understand when you show them, it makes the process easier!
I'm of a different opinion.
1. If you use the first person pronoun "I", you better make damn sure you wrote those words.
2. It's fine otherwise to use LLM words, but I beg you - DON'T SHIP YOUR FIRST DRAFT. Read it, reread it; delete words in it; replace three words with one; replace dumb or overused turns of phrase with fun metaphors; generally give it a sprinkling of joy; break up punchy sentences by making them run-on sentences to trigger your readers' OCD... you get it.
Did I mention not shipping your first draft?
Couldn't agree more with the post.
I wrote down a few of my thoughts a few weeks ago at https://notes.pault.ag/why-write/ -- I find myself scanning text to see if it's LLM before I actually engage with the content.
It is getting harder to do by the day, and has slowly moved from a mental exercise (look for specific markers) into a "gut feel" based on how the pieces fit together.
The issue with LLMs is not that they write poorly but that we all use the same models and end up with the same non-unique voice. This can come over as a combination of bland, boring, and unauthentic. And it doesn't help that some LLM generated text reads like these LLMs are biased towards a style of communication which embraces a lot of hollow phrasing, disingenuous language, overly enthusiastic language, etc. No surprise, the key companies are based in California in an environment where people that work there probably communicate like that. For them this is authentic. But I'm a non native speaker in Berlin and if I send language like that to my colleagues, clients, or peers, I end up sounding like a jerk.
It's something that is of course fixable. Either with a lot of prompting and direction. Or by training on existing text that you wrote. To some extent some tools already do this. But of course if you only ever let AI generate what you say, how would you even develop your own voice?
On a meta point, I did write all of this myself.
>Like many things, learning to be a good writer relies on lots of practice. It's a skill worth the time to acquire, but surrendering to an LLM will undermine any progress.
I assume that only applies to natural language, not computer languages?
In my opinion code works when it works so there is less reason to care about the weeds. When code is a communication of intent between people, or even just between a person and their future self, learning to encode that intent well into the code takes practice. There are entire fields people have developed over this, like domain-driven design or data-oriented programming, ensuring the data shows what it means semantically.
It is impossible to reconstruct the intent/spec from the code. More often than not the intent is not clear in the code, and one has to track down the original author to check their context and decision at the time.
As the saying goes, naming things well is a hard problem. It's kinda mystical in a way, where only by knowing an entity's true name will one understand it.
However if it's communicated between agents one may care even less.
This is an interesting distinction. I can't stand AI writing but have no problem with AI coding.
I think the key difference is that code is a tool used for a given purpose. It can be slop, or structured better or worse, but at the end of the day it exists to be executed, and its execution is either correct or not. Additionally, good code that does a given thing can only be expressed so many reasonable ways, probably. So LLM-generated code can be perfectly fine.
By contrast, human writing is about exchanging thoughts. Often those thoughts are about infinitely complicated and poorly understood systems, like other people, or emotions, or politics, so it's not a matter of exactly encoding something but attempting to describe it imperfectly. All the tiny little details encoded in how someone writes something can work to this end and are also a product of the writer's thought patterns and overall personality; this all conveys useful information to the reader. In other words, there is a much greater degree of fine detail that affects the quality of writing than of code, and LLMs suck at those fine details for most subjects that are interesting. On top of that, they don't have original thoughts.
That's only one side of the article's point though. The other, more consequential part is:
>Like most writers, I find writing an almost-essential tool for thinking. Often it's only by writing something down, that I can really get my thoughts on a topic into coherence. ... . As more people delegate writing to LLMs they will also delegate thinking and comprehension to LLMs.
There's an analogy with LLM coding that is so screamingly obvious I'm embarrassed to point it out (changing "writer" for "programmer").
I'm curious to know if the author, given who he is, thought of it (he had to, its just too obvious) and what he thinks of it.
Agreed. I generally find that I spend rather more time editing the names of things and the comments in the generated code, than I do the logic.
> I'm wary of them, partly due skeptism about their accuracy, but also because it rather misses the point.
Here, this reader wondered whether "to" was omitted intentionally to emphasize lack of LLM involvement. Then, on a second read through to find the sentence again, I noticed the first paragraph is also missing a period on its final sentence. Which just goes to show: we can all use editors and proof readers. (According to the acknowledgements section, nine humans read drafts.) Now, I don't want to intentionally leave typos in my writing, but for me it's preferable to LLM-voice no doubt.
> But I also would prefer rather a genuine human's clunky prose to the smooth but soulless LLM.
Amen. This reminds me of a bit from Kitchen Confidential about how Anthony Bourdain would prefer a simple and plain home cooked meal over an ornate restaurant meal any day of the week. I read because I want to hear what a human has to say. It's the errors and malapropisms and style that give a unique and human voice. Doesn't this contradict my earlier statement about not leaving errors in my own writing? It turns out I'm perfectly tolerant of errors in anyone's writing but my own. (I should probably bring that up with a therapist.)
There's a fine line between using an llm to write a personal blog vs using it to help you come up with an account plan on why you want to sell to a company that will be used internally.
> There are tools out there that claim to be able to tell the difference between LLM-written prose and the old-fashioned biological product. I'm wary of them, partly due skeptism about their accuracy, but also because it rather misses the point
HN uses automatic classifiers. They seem pretty effective. At the same time, I think it's a loosing battle. I'd rather HN either flag for others to ignore/filter/review if they want to, and/or allow people to self-report (ala TikTok)
Eg. I wouldn't mind if a comment on HN was like:
Hey everyone, I coded this little thing:
<ai-generated> some code </ai-generated>
Then users can choose if they want to see posts with AI-generated stuff or not, or something else...
Of course there are always going to be some people trying to game the system. But what are we optimizing for? (a lot of times the 95% unnecessarily suffers because of the mitigation efforts to prevent the other 5% from abuse)
Extremely relevant for about half or more of blog posts submitted here. Honestly has become a sad state of affairs
i noticed something similar yet different. because, sadly, most of what i read today is llm output (the chain of thoughts, or the summary at the end of a big chunk or work), i tend to "adopt" this style in my writing, hence, writing as an llm without using one. humans are an amazing creatures at copying stuff and adapting to our environment
I have wondered about this as well. To what extent am I influenced by the LLM output. It is a load bearing question for sure. ;-p
A billion humans use AI daily, they write about 0.5T tok and read about 2T tok (estimated). In a week they amount to more than the whole training set of GPT-4. Human language is being shaped back by AI language.
You hit the nail on the head
One reason it grates is that it is our brain's way of telling us that there is nothing to be gained from engaging the material; it is trite.
My workplace is so AI-pilled now that people have Claude responding to Slack threads on their behalf. It's not even discrete - the messages are literally tagged with a "posted via Claude Code" label, and unabashedly use LLM voice. Sometimes it's unclear to me whether the poster even understands what Claude wrote back for them.
It's infuriating to read, but even more so, it feels disrespectful. It's like they're implicitly saying, "I can't be bothered to respond to you, so just parse this AI slop".
I don't really know how to make a case against it since leadership is very pro-AI usage. Has anyone dealt with something similar and have suggestions?
I run startup so I set these standards in a small way. I aggressively promote AI use, but do not allow AI writing in Slack. Two reasons:
1. The writing is terrible to read and wastes time.
2. The act of writing ensures understanding. If no understanding is needed, then this doesn’t need to be in Slack, Claude could be just doing it.
I think the bit about "saying what you write" is what stands out.
I wonder if you took the pure speech to speech models and STT'd the output if it would remove the LLM voice? After all, can you even imagine hearing a human utter aloud some of the nonsense you see spewed by claude? Or give a longform presentation or speech in the form of the essays that are 100% LLM-written?
Maybe there's something in our human brains connecting oration and writing that influences the style of writing that pure-text engines just lack?
> But I also would prefer rather a genuine human's clunky prose to the smooth but soulless LLM.
And I wouldn't.
> 3: Michael Nygard commented: “There you really aren't trying to communicate thoughts from one person to other people. You're communicating with a process rather than a human. A human may open the envelope but that person is acting as a component in a cybernetic organism.”
But of course: manufactuting itself creates " productive mechanism whose parts are human beings, as Adam Ferguson describes the industrial workshop: "an engine, the parts of which are men".
I look at it like this: LLMs can generate code for those who know nothing about software development and they can generate text for those who know nothing about writing. Asking them to write it themselves will not get us better writing, but rather the opposite. If they were willing to learn, they would be doing that right now instead of using a tool that absolves them from learning. So, as bad as LLM slop may be, it's quite possibly still better than the alternative.
What tools or AGENTS.md prompts do you find effective for humanizing LLM output? Default GPT output is acceptable, but Claude is unbearable.
The “Simplified Technical English” fad was pretty effective, but now I just specify:
Just learn to like it.
I never use LLM to write anything, but unfortunately now whenever I am writing something, I am worried, am I sounding like an LLM? Using certain words, makes me worry, it will make my writing like an LLM, so I have started to avoid some words in my writing.
I really don't like this timeline ... sigh
Don't worry about those idiots, its a great filter in fact to get rid of paranoid "there is ai under my bed" types.
It's a little funny to me to read this from Fowler, given how enthusiastic Uncle Bob has become about agentic coding (albeit his loop specifies very strict code quality metrics). But yeah, I completely agree with the take. Fowler is, as they say, absolutely right.
> (it's not the em-dashes, it's the load-bearing tapestries)
I suddenly feel an urge to turn this into an image prompt.
I used to work on how my writing would come across and on writing what people would like. Now I spend a long time on how to cut the LLM voice out of what AI wrote for me (I keep repeating "this sounds too AI, fix it" lol)
> I'm also suspicious of my own ability to accurately recognize LLM voice.
With much sympathy for the headline this is the central problem.
It's not easy to say what Uncanny Valley looks like, and so we're left with suspicion not certainty.
BTW, I am now 4-5 out of hundreds of my comments since AI detection became a thing, called out for having "AI" tone. I fucking hate being objectified like this. It's not true, although I understand that assertion is hard to refute or support. All I can say is, some people write like some people believe an AI writes.
I'm big into AI but hate watching a youtuber when I suddenly realise their script is pure AI garbage, that they didn't even edit out the 'quietly's or the the kickers.THE KICKER is forever ruined. I hate it. The kicker can quietly get bent.
> In time it may be that LLMs will be so good that we can't tell the difference.
I have to believe that we are within a couple of years if not months of this. This problem is too well-known to not be a major focus of the frontier labs' efforts.
My own feeling is that this signal will perhaps never fully go away. Not in months; most likely not in a couple of years.
Not least because it’s getting worse rather than better. But also because as AI gets better at writing like humans, humans will change how they communicate to avoid being confused with it.
Which will get a lot easier once human writing is nearly extinct. Now that could happen in two years.
Yeah, I swear AI writing is worse than it was. Part of my big model release tests has always been asking it to write a few chapters in different styles, and I swear they have slowly gone from really bad, to boring and plain (I remember Gemini 3 being peak of this), to now over-complicated and quite bad. My guess is that they were initially being trained on writing quality but that has taken a backseat since it's not really a profit generator.
I think the signals also show up in different ways to different people.
For example, I don't dislike the way Gemma 4 writes so much.
But I am British and Deepmind's output somehow seems more compatible with the way British people communicate (it's a very multinational team, I suspect that is an influence).
Other people pick up on Gemma's AI writing tells a little more readily than I do. I have a greater tolerance for Gemma (and Muse Glimmer) than ChatGPT or Claude (or Qwen).
They're still heavily RL'd on response formatting/clarity, and against common LLM-isms. Long form prose quality is expensive to RL so I expect this to languish for a while until other low hanging fruit has been picked.
The trouble is that it's not an intelligence problem. You can make an infinitely capable LLM. It will still have a default voice.
The reason people notice "AI voice" is not that the AI voice is itself remarkable. It's that it appears so frequently. Every literate person spent nearly their entire life reading text written by a variety of different individuals. Dozens of different voices in your morning paper. Hundreds of different voices on every web forum. You get the idea.
Now, imagine that all of those voices were replaced by one guy (or two guys, or three guys if you're a Gemini believer). Everything in the newspaper is written by one guy. Every comment on reddit, and HN, and the NYTimes comment section, all written by the same guy. Other people tell him what they're trying to say, but he's writing all of it. Some people might even ask the guy to write in a different voice, but the voice is just that guy doing an impression of someone else, it's not actually someone else.
That guy's voice would probably start to get pretty grating. You'd start to notice every tell, every frequent word choice, every formatting quirk. You'd start to wonder if that guy was actually a good writer, or if having him write everything was just more convenient for everyone. You might even start writing like him yourself. This would be true even if that guy was an absolute virtuoso of the language and his writing was unassailable in a vacuum.
If everyone is using the same (small set of) model(s), then it doesn't matter if the model is GPT-3 or Claude Legendarium 9. Unless the operator - which in theory doesn't need to be a human - is doing something to move it off of its default voice, the outputs are going to sound the same, and you're eventually going to figure out, and get annoyed by, its tells.
most of the comments on this side are boring also
Lately, I stopped reading what people are sending. I'm asking LLM to visualise it for me and explain it to me in simple / concise / precise terms.
Do you mean in response to large drops of LLM output, or do you mean literally everything people send to you gets passed to the LLM?
Words come from machine? Work is of machine. I skip human and employ machine
Advocates of the dead internet theory are having a field day with this
I recently read DDIA for the first time, and I was surprised by how enjoyable it was after reading AI slop for a year.
Thank you for posting this. I don't have a lot to add on the subject, but I found it interesting and thought-provoking given the age we live in. I find it really hard to tell real human and LLM text apart, even though some people claim they can, calling it "AI slop".
If it puts an end to LinkedIn-speak I'll take it, for a year or two.
I think the key tell is that the writing doesn't actually illuminate anything worth learning. The more I see of AI produced output, the more I believe that there is nothing there that wasn't already in the prompt. It's an inflation of a single idea to pages of text. It meandors about and never actually gets into detail.
It's a lot like talking to someone who doesn't know what they are talking about but are good at faking it for a while. The game works by leaning on pat phrases, circular reasoning, and hoping nobody is paying too close of attention. I think this is why it's so popular with the corporate executive class, as that has been their mode of communicating their entire careers. To them, they don't see anything wrong.
"Shitty writing existed before AI" is unfortunately an all too common rejoinder to this issue. That's not an excuse. That doesn't make it better. Just because the pointy haired boss talks that way doesn't mean we should be gleefully pouring gasoline on the fire.
I noticed the exact same thing, and I also don't want the LLM to suggest anything to me. I do want it to comment, and to tell me what could be improved, but not to give me text.
I created a small tool to make this easier: https://www.writelucid.cc/. It's free (bring your own API key), feedback would be appreciated.
This guy sure likes to use his own dashes, just saying.
I disagree with the advice to avoid having an LLM write a first draft and then editing it yourself.
To me, for most business and academic content, the ultimate goal is to convey information. If I can edit AI copy to legibly convey that information clearly, that's enough and I can move on.
The saying, "Don't allow perfection to be the enemy of the good," comes to mind.
And I'm not convinced an individual can accurately edit without learning just as much as if they wrote the content. I guess the question is: how hard does the person work at editing. And does the individual challenge themselves to not put out content they themselves don't understand.
We're in a transition period. And as in all transition periods, many recoil at what is new. LLM writing still feels strange. And it's understandable.
But, for better or worse, in a business and academic setting, as the years go by, most of us will shrug and move on.
As with any topic, there will always be a minority of fanatics who cling to the rejection of AI writing in all its forms. Just as some prefer Vegan leather or tabs over spaces.
But among those who get things done, most (though not all) will accept that it's not worth it to exclude AI from writing. There simply isn't sufficient ROI.
It's like trying to bring back low-end highly human employed manufacturing to wealthy industrialized nations.
That ship has sailed.
I do agree, however, that in cases where an individual is reading for pleasure -- e.g. not in a business or academic context, authors will be forced to pick their audience: those who enjoy writing, even if it smells of AI, and those who don't.
I guess to summarize, my feeling is that using LLMs to write is perfectly acceptable, and used well you'll likely learn just as much. But you must be willing to lose some of your audience. And, over time you'll lose less and less of that audience.
I don't know, with LLM it feels like I finally have possibility to prepare all the dull text that corporate environment expect in a form that colleagues are able to digest even if they don't specially enjoy it like it's the best poetry their beloved one ever composed specially for them. This is in sharp contrast with the struggle to shrink into a somewhat coherent excerpt the wide web of interweaved psychological experience flow graph into this alien expressions which are text streams. It's still impossible to dump really the whole experience and communicate wholly the full semantics they cast, but at least these automatons are never scared to face far much than they can sustain in what can be thrown at them, and apparently most people prefer the dull soulless prose to whatever can suggest vivide continuous reconnection of everything as the context inevitably drift and everything needs to be reconsidered properly with all the rest.
It's incredibly hard to maintain your own voice while using LLMs. It requires deep discussion with the LLM itself so that it comes to understand your perspectives and also explains to you the premises on which its own recommendations are based.
I haven't really done a systematic analysis to compare the quality of fully human-written content and content generated via this kind of deep human-AI collaboration, but my intuition is that the latter is vastly superior. Perhaps AI at its best is a research tool, but a genuinely intelligent one that we need to engage in conversation with in order to use to its full potential.
The sense that I get is that as a conversation gets more in depth, the LLMs start behaving more like reasoning engines and less like statistical averaging machines. The former is vastly more useful to a person who wants to develop an idea and articulate it well.
> But if I want to communicate with humans, my humanity has to shine (or at least ooze) out of my prose.
Yeah. I want to be able to touch a human being (or at least have one touch me) through the words. LLMs poison that, even if a human prompted the LLM.
Translation... I think I object to translation less than Martin does. There's a human back there, and they're writing all the words, and then an LLM is translating them. The LLM may not translate them perfectly, but they aren't going to introduce "load-bearing" when it wasn't there.
The bit about using LLM when you're writing for a bureaucratic process rather than a human was interesting, at least to me. I hadn't thought of it that way before, but I think it makes sense.
Yawn. This subject has been discussed ad nauseum. The lines are drawn. I'm over it.
So many "load-bearing seams"...
People have been talking about the importance of curation for more than a decade.
Because many good ideas are not immediately obvious good ideas, in an environment at risk of information overload, we need some way to quickly vet ideas to see whether they are worth investing the time to consider them. (It is often the case that the best ideas require considerable investment of time and effort to be judged as good, and, it's also often the case that this determination can be made only at the very end of such a process.)
“Don't judge a book by its cover,” sure, but, historically, there have been superficial markers of care and attention that we've used to these ends. These are obviously very flawed signals, but, you know, maybe if you took a lot of time to craft a message, I don't feel that bad if I waste a bunch of time trying to decode it. These markers—attention to detail (e.g., spell and grammar checking,) clear-structure (e.g., “bottom line up front,”) and typographical flourishes (e.g., smart quotes and em-dashes and extended unicode like diæreses…)—were an indication of some modicum of quality control and care. I, myself, used to brag about or show-off having set up .Xcompose keybindings (https://wiki.archlinux.org/title/Xorg/Keyboard_configuration...) which was just rare enough across the general population to be noteworthy.
All of this that was once somewhat expensive is now cheap or basically free.
Even if I make a point of going back to a pre-UTF-8 Usenet style (or even intentionally insert run-on sentences or misspellings or grammatical errors,) the cost of an LLM to style-transfer its output to match this is barely much of anything. All it means is I wrote one prompt for the first draft, and then one more prompt for the refinement.
Additionally, some LLM habits are actually good habits. Putting the bottom-line up front (in bold, even!) is a great habit in a business e-mail! Decision-makers genuinely struggle with reading through long paragraphs of detail!
Unfortunately, the problem with the LLM output is that it takes good, valuable, superficial or structural habits that it can cheaply apply, and it drapes those around inane, banal, cloying, muddy, meandering meaninglessness.
Today, we can maybe still judge a book by its cover, because even one more prompt worth of refinement is still too much effort, but I don't think that's amounts to much, since the frequency and loudness of complaints of LLM writing means that model trainers are going to train their models to adapt to whatever signals we use for quick vetting. I am skeptical whether this training will concurrently improve the novelty, innovativeness, or other intangible qualities of the actual message. (With the models, as they exist today, and as we use them today, at least in the field of software development, I see extremely poor capability at generating innovation… Personally, I don't even like the quality of the Python code they spit out, but they sure do type real fast, and, like, pejoratives aside, unexciting, mediocre, corporate-y code does have a real place in the world.)
So what happens when the economics of production in the medium significantly outpace the capabilities of production in the message? The LLM can write a comment nearly indistinguishable from any given human… except when you actually take the time to read through it, it's just inane drivel. Thoughtless and a waste-of-time. And there's basically no way to manage your investment of time and effort to tell one way or the other.
You know, maybe it was never the case that superficial markers of effort ever meant anything at all. Maybe there never was a way to tell from the envelop whether the letter is worth reading… and it was confirmation bias all along. Maybe there really are no shortcuts to judge whether an idea has any merit…
Because, for sure, some of the real bangers of comments I've posted on HN and similar forums—all my XCompose-configuration aside—have gotten relatively few upvotes!
Same thing with conference talks. The ones with half a million or a million views on YouTube… the real magical ones, I think, are the ones no one seemed to pay any attention to… e.g., https://www.youtube.com/@PyDataTV/search?query=james%20powel...
[dead]
[flagged]
An important distinction as AI-generated content becomes more common: having a consistent voice is not the same as having a distinctive point of view. The best AI-assisted writing should preserve the author's personality, judgment, and expertise rather than simply imitate their vocabulary or sentence structure. This is a useful concept for anyone building content workflows with LLMs.
[flagged]
[flagged]
Great piece. The "LLM Voice" problem is real — we've been benchmarking 18 free LLM API tiers (Qwen3.8-Max, GLM-Z1-9B, Hunyuan-MT-7B, bge-m3 embeddings, etc.) for consistency and tone variation across providers, and the homogenization is striking. Models from different families converge on similar cadences and filler patterns even at different parameter counts. The one outlier we found: reasoning-focused models like GLM-Z1-9B actually break the mold with chain-of-thought traces that feel more distinct. We publish the live benchmark data at apishare.cc/leaderboard if anyone wants to compare latency/quality tradeoffs across free tiers. Would be curious if others have found specific model families that maintain a more distinctive voice.
[flagged]
[dead]
Journal entry: Dec 12th, 2027.
My name is Martin the last human organic writer on Earth. I am better than everyone else; my citations in my 4-paragraph article say so.
personally, i'm for LLM and AI usage in writing if it helps the writer communicate ideas, their stories, and triumphs in ways they may never have been able to before. I just had one of my posts removed from the front page here which was disheartening because it feels as though what was judged was prose rather than the technical insight in to a niche problem, which seems counterintuitive considering i've always thought of Hacker News being an inclusive place for the technical and engineering skills rather than their level of the english language.
> I just had one of my posts removed from the front page here which was disheartening because it feels as though what was judged was prose rather than the technical insight in to a niche problem, which seems counterintuitive considering i've always thought of Hacker News being an inclusive place for the technical and engineering skills rather than their level of the english language.
The problem with LLM voice is that it sends the signal that it's a likely waste of effort to try to figure out if there's any "technical insight" there, because more often than not, there isn't.
Maybe it's not fair that you're paying the price for others using LLMs to abuse their readers, but it's the reality that you have to deal with. It's also not fair to expect readers to wade through all that abusive LLM prose in order to find your (assumed) non-abusive post.
Once LLM writing becomes indistinguishable from human writing, it will be really sad. There will just be a collapse of human engagement, and you might as well just talk to your machine because then at least you know what you're getting and aren't being deceived.
in some ways I agree, however I think that speaks more about the reader themselves. Just like when communicating with someone who's first language isn't english, I don't disregard the meaning and intent behind their words simply because the way they communicate isn't formally correct. Regarding my post that was removed, it was lovely to see the comments of people that had enjoyed the technical write up. Me personally, i'll always value an attempt at engaging and communicating than none at all.
Good lord, someone not being good at english does NOT mean that they put not effort or care into their attempted output. It is RACIST to make that association and thats why its an obvious faux pas.
An LLM prose article ABSOLUTELY can mean that. To attempt to conflate the two things is maliciously misleading.
This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.
Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
> This is an interesting corollary to a thing I'm seeing where "devs who thought their job was writing code" are in like existential danger.
I think you're misunderstanding that anxiety. I think this is what's going on:
1. For a lot of people, using technical skills like coding is the part of the job they enjoyed.
2. Closely related to the above point: for a lot of people, technical skills was the part they were good at.
> Technical skills aren't worth shit and never were, except as far as they enable you to engage with other people in ways you and they find valuable. This used to be really far! It made it easy to believe that the technical skills on their own were what was valuable.
> But the engagement was always the valuable thing, generating technical insight was a necessary prerequisite for it. Now anyone with a $20 claude sub can generate technical insight into niche problems, so what's left is everything else. Everything of value.
3. More broadly: if "they" don't find the "technical skills" you provided valuable anymore, why do they need to keep you around? Why not cut your 10-person team down to 3 people (the manager, architect, and technical lead) and fire you and the other 6? Not everyone can re-invent themselves as a product person, even if they had the skills. It's a game of musical chairs.
No I get it, I didn't mean to be dismissive sorry.
It really is ultimately a labor relations problem. A lot of people thought they were guaranteed to not face penury and now they're realizing they may have been wrong about that. The issue is that it's ultimately not under your control whether you can sell your labor, and there's no universal fallback if you can't. Solidarity and concrete power for workers are the answer to this and I don't think there's another one.
For the rest, yes, certainly. The vast majority of people do not enjoy their work, are not valued or rewarded for the skills they apply to it. Again, if you thought your job was writing code this sucks. But you're merely one of the billions now. I understand the grief for what people are losing but very very few people ever had that rare privilege.
I find a lot of LLM writing exhausting to read. It is like I have to exert extra energy to attenuate my BS detector to cut through what is actually being said, and why.
Imagine you cooked something great, a deliciously tender cake or something. I ask you if I can try some, you say sure, cut off a little piece for me, then you drop it and tell I can have it on the floor. That is what AI is doing for writing. Why should I care how the cake would taste, if I don't like eating on the floor?
i don't really agree with that view, but i respect your opinion. In my view, i see it more like someone who's never baked a cake before, offering you a slice, and whether or not it's good may be up to you to decide.
AI doesn't help the writer communicate ideas. It helps them obfuscate those ideas behind bloated language where even the writer can't discern if it's good or not.
It's the modern equivalent of going to a thesaurus and replacing all your natural words with five-dollar ones.
If you're not confident in your English then just write your brilliant ideas in broken English. Or at most do a direct translation. But don't fluff it with AI's nonsense.
If the ideas are good they will be recognized.
Assuming the author had to input good points for the LLM to expand it into a full length post, just give me the points. Give both formats if one must. Translate, spellcheck, fix mistakes with an LLM. Don’t inflate 3 bullet points to a 10 minute read.
I can’t distinguish between well meaning slop and blogspam slop. I might be able to if I’m well informed on the topic and can point out inaccuracies. Otherwise I just don’t know what is accurate and what is filler or outright wrong. LLM-isms are very distracting and exhausting to sort through. It just doesn’t make for a good reading experience.
Yes, this. I mean, if you're writing a novel meant to be read, maybe not. But if you're just trying to communicate 3 ideas, write them out in a way I can quickly tell if it's worth reading more.
There are few things (relatively speaking, I mean online activities, etc) more annoying than seeing a click-baity headline, and then having to read 17 paragraphs to actually get to the point that I realize I already knew and understood it, and it's just slop padding.
Just do what I do: capture everything I've ever written and ask AI to cobble together what I said using my own sentences.
No more LLM voice! Every now and then you might get a nonsensical artifact or two, but that's the price you pay for better groceries.
"Ya i literally pulled this meal together from EVERY dumpster in the city. Thats the price i pay for better meals"
I'll repost a comment I made a while back describing the process I use, which I think balances AI in keeping your voice.
If you care about your voice, don't let LLMs write your words. But that doesn't mean you can't use AI to think, critique and draft lots of words for you. It depends on what purpose you're writing it for. If you're writing an impersonal document, like a design document, briefing, etc then who cares. In many cases (scientific papers, legal documents) you already have to write them in a voice that is not your own. Go ahead and write these in AI. But if you're trying to say something more personal then the words should be your own, AI will always try to 'smooth' out your voice, and if you care about it, you gotta write it yourself.
Now, how do you use AI effectively and still retain your voice? Here's one technique that works well: start with a voice memo, just record yourself maybe during a walk, and talk about a subject you want, free form, skip around jump sentences, just get it all out of your brain. Then open up a chat, add the recording or transcript, clearly state your intent in one sentence and ask the AI to consider your thoughts, your intent and ask clarifying questions. Like, what does the AI not understand about how your thoughts support the clearly stated intent of what you want to say. That'll produce a first draft, which will be bad. Then tell the AI all the things that don't make sense to you, that you don't like, just comment on the whole doc, get a second draft. Ask the AI if it has more questions for you, you can use live chat to make this conversation go smoother as well, when the AI is asking you questions, you can talk freely by voice. Repeat this one or two more times, and a much finer draft will take shape that is closer to what you want to say. During this drafting state, the AI will always try to smooth or average out your ideas, so it is important to keep pointing out all the ways in which it is wrong.
This process will help you with all the thinking involved being more up-front. Once you're read and critiqued several drafts, all your ideas will be much more clear. Then, sit down and write your own words from scratch, they will come much easier after all your thoughts have been exercised during the drafting process.