These are all important points and I love the analogy. But there is an even bigger issue with having LLMs write for you:
Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
Reviewing gives you the chance to ensure the arguments connect solidly, that references are accurate (even informal references) and gives you the time to consider counter-arguments you aren't addressing.
None of this matters much on LinkedIn, but it matters a lot in our work. You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
> Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
I don't disagree, but I think it's often not appreciated how much there's other work to writing too.
The biggest one is that you have to communicate non-interactively to an unknown audience. Having to (literally) put it in someone else's assumed terms does help giving different perspectives into the matter, but doesn't necessarily help one's own thinking that much. Instead you have to do some of the reader's thinking for them.
You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Not all writing is thinking and not all thinking is writing.
Strange! I would quite specifically highlight “having to do the reader’s thinking for them”, as well as more generally developing the skill to “communicate non-interactively with an unknown audience”, as extremely valuable upgrades to my thinking.
It's a kind of shared fiction of an imaginary person's thinking, really. The author writes "now, I know what you must be thinking", and hopes the reader will agree "OK, close enough".
Most valuable thinking is done at the margins, where you don’t have much capacity to emphasize with a diverse and unknown audience.
That said, abdicating to an LLM is the worst of all worlds - you’re not thinking and the product is not tailored.
The solution is obvious - write as much detail as you need and allow readers to interrogate the virtual you with an LLM, maybe not even reading what you write.
> You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Hard disagree. Constraint is the driver of creativity. Also rewording sentences to sound better or make sense can make you reconceptualize the whole concept you are expressing
Sure, but doing an interpretative dance or an abstract painting can make you reconceptualize the whole concept too. But we're not really pushing those tools as much as writing.
Take for example a non-native writer of the language. I'm sure having to check up words from a dictionary may help to reconceptualize things, but I'm sure also that it's not often very efficient. And I think similar is going on for natives too for many types of writing.
It’s not clear that doing an abstract painting can make you reconceptualize in the way that reframing in other words does. The point about putting in other words is that you may stumble on a clearer, more tractable, more extensible framing. The kind of reframing an abstract painting does is very different, more like changing your attitude or way of looking. But I don’t think it ever leads to a place where you will suddenly find yourself with a sharper understanding that helps you communicate with others more effectively.
I'm not much of an artist, but I'd guess trying to paint a painting of how quicksort works, in a way someone else can grasp it too, can need a lot more reconceptualization than writing a description about it. Doing plots or diagrams, or even implementing an algorithm, for sure often need more thinking than writing a description.
Yes but diagrams aren’t abstract paintings (or interpretive dances), which I assume the original commenter meant as intentionally artsy and indirect forms of art.
> Writing helps us think about the world, it’s a pivotal intellectual technology.
Much like money decoupled selling and buying to move away from bartering, writing decoupled saying and hearing so they didn't have to happen at the same time. The incredible step that happened was not that people had to think a whole lot, it was that thinking that was already happening had to happen once.
> All writing is thinking when done by a human, you’re literally distilling your thoughts into words. You can’t write without thought.
Of course you can. You can write down exactly what you hear, for dictation.
You can write down a stream of consciousness and put barely any thought into it at all.
I can't help but feel most here are massively over estimating human writing. Human writing is, almost universally, terrible. We have entire jobs that are hard to fill just to make things sort of ok. Good writing is a small subset of human output.
This is an example of the problem. I did choose to write it in the way that although I knew it can be interpreted trivially, if the reader assumes I'm an imbecile. It was a "punchy" recap of the relatively long explanation for a HN comment and hopefully decently argued point.
I have been thinking over this comment for a while now. I think yes, you are correct that thought is involved in writing but I don't think it is possible to claim "all writing is thinking when done by a human". Like what type of thinking are you claiming here?
Because one can copy a text and write it down and that involves thinking in the sense that anything we do involves thinking fundamentally. But that thinking is different from thinking logically about a concept and writing it down which I think is where you are getting at.
The definition of writing and thinking is too broad in that sentence even though it does apply in several obvious cateogires within that at different levels.
And also "writing helps us think about the world" is too broad again. Why? Why does me writing "apt apt apt apt apt apt apt" help me think about the world? I just wrote it because i felt like writing it. Why wouldn't you consider that writing?
Yes, but not to understand or think about the world. In this case, writing was to express something, not to understand something. The overall point is that claiming that "human writing = human thinking" is too broad a general statement to make seriously.
You wrote "apt apt apt apt apt apt" to prove a very specific point no? Absolutely requires reasoning and understanding the problem to go there. And good luck getting an LLM to do that.
Yes, I wrote it to prove that writing doesn't mean I understand the world. It responds to "writing helps you understand the world." My point was that, yes, of course you can find categories within it that apply, which is why the answer is not fundamentally incorrect. Claiming "writing = thinking" in a general sense is very broad, and that's my point.
Bad example, because translation is deeply creative. You can't blindly mill one language into another, because words and phrases and concepts and cultural references in one language frequently don't map 1:1. You have to find a way to convey meaning as closely as you can and not necessarily the words.
I think you meant that translation is, case in point, based on inference. The person doing the translating is inferring, based on context, intent and meaning and imparting that on what their output is. That's generally not desired from a taxonomy perspective.
Amusingly I absolutely abhor translations done by people who think the way you do. I want a translation that's as literal as possible and which provides the necessary commentary for me to understand any alien concepts, idioms, customs, etc. I absolutely never want "translated" cultural references. At that point the "translator" is nothing more than a shitty fan fiction author as far as I'm concerned.
Of course the above requires actual work on the part of the consumer. I realize many don't want that, particularly when it comes to entertainment. So I appreciate that the other sort of "translation" exists but I think it's important to realize what exactly those are.
Thankfully LLMs are more or less to the point of providing what I'm after in near real time.
But you aren't disagreeing with your parent comment. You seem to have a strong opinion that you are; that opinion is incorrect.
Words refer to a broad semantic region, a phenomenon technically known as "polysemy".
The range of a word in one language is always different from the range of analogous words in another language. This is a classification problem. And a translator must think about how to solve it. Imagine a Venn diagram with 20 circles that each overlap the other 19 to differing degrees. What does it mean to designate one of those circles as "the literal translation" of a foreign word?
Of course nothing involving natural language and human culture is exact. A "literal" translation is obviously a slightly fuzzy concept that speaks to intent. However I think I provided enough context that this should be clear. I gave the example that I don't want "translated" cultural references or idioms but rather the (approximately) literal wording and some commentary from the translator providing the necessary context.
Obviously there are degrees to this and obviously preferences will vary. I acknowledged that.
I think you've misunderstood something. The comment I replied to specifies "words and phrases and concepts and cultural references". You only seem to be talking about words (and perhaps exceedingly simple turns of phrase). I was quite direct that I disagree when it comes to (among other things) the more complex idioms and certainly when it comes to any and all cultural references.
Imagine a localization attempting to replace a reference to an actor, political scandal, or other concrete cultural reference from one country with the "equivalent" from another. I've encountered that sort of thing before and while there are certainly those who appreciate it I am emphatically not one of them. As far as I'm concerned that's shitty fan fiction.
There are also a lot of examples in most (all?) languages that rely on repetitive sounds, easily mistaken words, or other strictly auditory features of the native language. You literally cannot translate those things. I do not want shitty fan fiction, I want an explanatory note.
- You're going to rant about it, because you want to, whether or not it's relevant to an existing conversation.
- You didn't bother to think about my comments.
- You didn't bother to think about habinero's comment either.
Here is the same passage of the Analects (part of the chapter Gongye Chang) in different translations:
--- Annping Chin ---
Zilu said, "We would like to hear what you would like to see yourself accomplish."
The Master said, "To give comfort to the old, to have the trust of my friends, and to have the young seeking to be near me."
--- David Hinton ---
Adept Lu then said: "No Master, we'd like to hear your greatest ambition."
"To comfort the old, to trust my friends, and to cherish the young."
---
Our focus here is on the second line, what Confucius says. Does he want to trust his friends, or does he want his friends to trust him?
Does he want to cherish the young, or does he want them to cherish him?
We might also ask, though the translators have agreed on this point, whether he wants to comfort the elderly or for the elderly to comfort him. (And we could further ask whether Confucius wants to personally comfort the elderly, or whether what he has in mind is for society in general to do that.)
All three clauses are formed the same way in the original Classical Chinese, and for a couple of interacting technical reasons they are all ambiguous in this way. Translators, as you can see, make different choices.
But of relevance here, when you're doing a translation to English, you have no option but to make a choice. It isn't possible to render the original text 'in literal translation' and append a note explaining what went wrong. You must commit to a meaning behind the text and phrase that meaning in English. You can also append a note explaining that you might have chosen wrong, but English simply doesn't allow you to do anything that parallels the source material.
I really think you aren't talking about the same thing that I am. I'm not sure why you're assuming bad faith on my part rather than engage in discussion to clarify.
I think it should be quite clear by now that I am not talking about isolated words that broadly lack an equivalent concept in the target language. I even quoted the bit from the original comment that I took issue with and proceeded to give examples so I'm really not sure where the misunderstanding between us could lie at this point. Perhaps you are the one who should stop and more carefully think about what I wrote?
As to your example. I certainly do not accept that this is a case where we should throw our hands up and accept that different translators will go about things differently. Those two sentences in english have (as you note) rather different meanings. So either one or both translators must be wrong.
You have indicated that the original work in the native language is ambiguous. In such a case I do not think it is remotely acceptable for a translator to arbitrarily pick one of several possible meanings and just run with it. If the original meaning of the text is ambiguous then removing that ambiguity changes the meaning thus it is a bad translation. The translator instead needs to faithfully communicate that ambiguity, possibly resorting to a note if it isn't possible to easily express such a thing in the target language.
I realize that many people aren't going to want such a marked up copy. But without all the gory detail the reader will be consuming some sort of bizarre partial fan fiction. Your example illustrates that perfectly.
I did think about this while writing, and I made the compromise to accept that someone will nitpick about it to hopefully drive the point better for those willing to read it charitably.
What is the charitable interpretation? I am not being coy or sarcastic here; it is genuinely (forgive the claude-ism) unclear to me what your arguments are.
With the last line I tried to condense (and oversimplify) about these ideas:
Not all writing is thinking: That all, even a lot, of writing, or parts of writing, is such that it will develop one's thinking much. For example most stuff I have to write, the dozen emails a day, the funding application boilerplates, the reports are stuff that don't really need (or deserve) much thinking but they have to get written. And even in the writing that deserves attention, there is stuff like grammar and spelling and surface style that usually take quite a bit of time after the ideas have been written down already.
Not all thinking is writing: for many cases writing is not a particularly efficient way to develop one's thinking, and e.g. visualizations, math, coding, discussions etc can be a lot better.
When you transfer other people's thoughts. On this board, we are engaged in pursuits where the truth matters. A lot of writing is about showing that you belong to the right in-groups. That is better done by repeating their talking points than sitting down and coming up with an earnest way to show that you agree.
The number of times the friction of writing has saved me from prematurely communicating a poorly understood idea must be in the hundreds or maybe thousands. For me when something is difficult to write about, it's a very good signal that I don't understand it well enough. So I think all the things you label as aspects of writing that aren't necessarily "thinking" are nonetheless good for thinking because they provide some necessary friction.
Hence The Great Smoothening of Minds we're all experiencing in this decrepit era. I blame the financial incentive, and welcome its disappearance. There are too many people doing computers just for the big paycheck, it would be more fun without them. Maybe the AI bubble popping will get rid of them.
(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
It reminds me of the transition over the last year from AI-assisted coding to AI doing all the coding. At first the code output wasn't good enough, and humans read and iterated on the code all day, so the details of the source code mattered. Now, the code is largely high quality and it meets a large set of guardrails we've set up over the years (linters, typecheckers, security checks, LLM-assisted code quality checkers), and it's just Claude working on the code, so the details matter less and engineers think a level or two up (machine code < assembly/bytecode < source code < conversation with agent < artifact with high level design).
I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document. But writing and coding are different enough in a number of ways that this is far from inevitable.
> wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
What use is that? I'm not being facetious, I'd really rather like to know.
Who or what is the audience for that sort of long form writing? If it's a human, why would they read it? They'd just give it to an LLM and get the salient points back. If the audience is another LLM, why expand it?
The only use case is an audience of humans who still read and understand, and those people aren't going to be interested in a message when it is not apparent that the sender actually understands the message themselves.
A lot of corporate documentation exists solely to measure if people are working or not.
That's why there's so many meetings in white collar companies. Because people can't understand what is going on at those documents so they just need to "align".
LLMs are amazing at generating this useless documentation that goes absolutely nowhere.
1. I have a bunch of data or research that I've gathered with a unique hypothesis
2. Having gotten my arms around that pile of information, I believe I have a compelling thesis to put forth
3. I design the narrative arc and of the thesis. The important parts, the necessary but not sufficient scaffolding.
4. An AI helps fill in the story from there. Fact checks each claim, connects the dots, makes it comprehensible.
Who is this for? Well, quite possibly the human who asked for it. It's pretty informative to read back a research brief in full that you helped do the scaffolding.
Also of very clear use is other AI's who did not have the same unique hypothesis and did not gather the supporting evidence. It's an interesting angle for others to build on.
And of course, other humans! Most human written content gets almost zero readers today as it is. And I suppose LLM content probably pulls the asymptote closer to zero, but some pieces of content may be genuinely interesting or useful.
> An AI helps fill in the story from there. Fact checks each claim, connects the dots, makes it comprehensible.
I think this certainly has some value but this claim in and of itself is stated like your hand-wavy step 3. How do they fact check claims and connect the dots?
Maybe LLMs get there but currently they write in an extremely verbose manner, and things that have gotten into the context window that are no longer relevant continue to stick around (just try having it write some code, then work some of it back to simplify the problem - it will insist on writing comments about code that no longer exists).
Right now using an LLM to write documents is like taking a superhighway to travel 100 meters. Yeah you're doing a lot but is all that really necessary?
I won't deny that LLMs will never have a place in writing. But I personally don't think the current form is "the one that actually lands" (!).
Then the reader can use an LLM to compress it back, and you can then interrogate it for details.
I used to say this was the future of advertising (cr sales person prompts “we have some new EV SUVs on the lot”; GPT generates an ad email with a synthetic video, blinking text etc; then the recipient’s spam processor tells them “that dealer has some new SUVs”. I suppose the same could happen with so-called “long form”.
> What use is that? I'm not being facetious, I'd really rather like to know.
People are terrible at writing. Near universally bad. Even good writers have drafts and editors.
There is a constant refrain here that somehow short messages are more valuable than longer ones. But that assumes it's understandable. Lots of short content is, frankly, awful because the writer cannot put themselves in the position of the reader and explain all the things around the point they're making that the reader really should be told.
You can view writing as translation. From your language to a language your audience speaks. At that level is it so odd if the word count differs from one side to the other?
Not necessarily. Poorly worded, ambiguous, confusingly ordered writing can be massively improved without changing the core content. Better setups and explanations can be longer without changing the message or meaning.
Look at it the other way, could you take a good longer message you’ve written and make it shorter and less readable for your audience while still making sense to you and containing the key points?
But this is the same with coding. The reason that AI can write code from a description that is shorter than the output is in large part because it makes decisions about the behavior that were unspecified in your prompt. We accept this for coding apparently, I guess because those decisions are often unimportant. We might accept it for writing too. I hope not.
wasn't it thoreau who said "Not that the story need be long, but it will take a long while to make it short"
personally, I think there's a time and place for short versus long, just like there's a time and place for a 45mins TV episode versus a 2 hour marathon movie.
You claim writing is thinking, but imply writing code isn't thinking.
My opinion of LLM design review isn't that high - it seems to miss design tweaks that could vastly simplify corner cases. But if your code isn't written for human consumption maybe it doesn't matter. I'm still directly responsible for what I commit, so I can't just offload it to Claude.
That's something I struggle with, I try to get LLMs to output code I don't care much about and focus on the parts I do and it kind of works but the problem is that reading code written by the LLM is even worth than reading LLM generating text. It's nauseating and you still have to read what the LLM did if you want to really work on the parts that matter.
People have been "writing code" like this for decades. That a programmer happens to do it doesn't turn it into writing code, no one would have made that claim 25 years ago, and people who aren't programmers wouldn't make that claim today.
It's like watching somebody about to be hit by a bus. You yell, you wave your arms, but they either don't hear you, or they don't believe you. The last thing that goes through their head is a Greyhound's hood ornament.
There is no subtext to bigstrat2003's argument, other than "I'm wrong, and I don't care." If you give persistent, repeatable instructions to a computer, you are programming it. If you disagree, you are gatekeeping. It's that simple.
The most popular programming languages in 2030 will, in fact, be English and Mandarin. Deal with it and get over it.
----------------
Edit, to bcrosby95: Look up the etymology of the word 'computer'. It didn't originally have anything to do with hardware. The first computers were people, who were told what to do ("programmed") without necessarily knowing what they were working on in a big-picture sense.
There is subtext to my argument, namely that you're the one standing in front of the bus, and trying to pull the rest of us in front of the bus, while shouting at the people standing safely out of the road. I'll continue to reject the replacement of humanity. (That doesn't mean remaining ignorant of AI as a technology. It means rejecting the idea that we should be thrilled to be replaced by AI.)
LLMs do not produce repeatable results, by their nature. Two people can give the exact same prompt to the exact same LLM and get different results. It's not a straightforward 1 + 1 always equals 2 process. This is like telling someone else to code something for you. You relinquish control of all the details.
Calling it gatekeeping is just laughable. That's like saying it's gatekeeping to say that the painter painted their painting, and that the person who commissioned the painting did not paint it. It's wholely absurd.
Anybody can pick up a book and learn to actually code themselves. Or you can use an LLM to try to make things without bothering with that. But even if the LLM worked perfectly, pretending these are the same thing is silly.
The history of the word computer is obviously irrelevant. Words change, it turns out.
It isn’t really by their nature that they don’t give repeatable results, right? They’re just a bunch of math, but they perform better with randomness injected so we choose do to so.
(I’ve heard about some GPU compute nuance meaning that even without randomness injected they still wouldn’t quite be deterministic, but that’s also not core to their nature)
I'm game. Say $1000, donated to a charity of the winner's choice? How do you want to set the bet up, and how do you think it should be decided?
To be precise: I will bet that high-level programming languages won't be any less popular as a whole, but the vast majority of code will be written by AI rather than humans, working from specs written in natural language or something very close to it.
What we call "source code" today will be thought of as "object code" by 2030. Something that occasionally needs to be inspected by humans, but rarely authored directly. Anyone not writing code this way had better be doing it as a hobby, because almost no one will pay for it.
Other person probably doesn’t like the idea that LLMs will replace hard earned skills. On the flip side, I bet you’ve seen your skills atrophy at an alarming rate and are trying to justify it.
Both sides come from fear. Just relax and take things as they come. Whatever happens happens.
Expands into a document for _who_ to read? another LLM to re-compress?
It's baffling you people are in control of such a strong product when you are obsessed with this intellectual pornography; wow - look at how smart it made my thoughts look (n.b. look, not read). Don't look too close. And certainly don't ask me what it means.
You are just talking about the output though. If you only think at a "higher level" you aren't doing the actual thinking. Its the same with code. The output may be good enough, but over time you lose touch with the details to the extent that you can no longer serve a useful steering function for the organization. Before coding agents I'd seen this with many humans when they get promoted passed the point where they work with code directly and can't figure out how to add value there.
Many programmers don't write long text; their way of getting a deep understanding of a problem domain is to build something, is to write code - in a process very similar to writing a long piece of text - it has the same reflection and externalization of thought.
The widespread introduction of LLM code generation is very destructive to that.
Perhaps LLMs can be brought to support human cognition in the same way writing can; but that has yet to be designed and it does not seem to be the way things are heading.
Miss me with this. I write way more now than a year ago. Mainly because I have to explain myself to the LLM. I'm fairly certain I write 50x more than before simply because before my weekend was spent gaming and watching anime. Now I'm having fun building things.
LLM prompts definitely don’t feel like “writing” to me in the way that writing a blog post (or even an HN comment) does or writing code used to. There is nothing to work out, you don’t really need to think. You’re _typing_ sure, but I don’t think it counts as writing maybe because the text that you write is thrown away.
I am doing memory research on emotion, sensory perception, and cognitive quality over time while explaining in extreme detail how to do a specific hinge animation in another thread doing blender animations for mechanical movement. I am also writing blog posts manually about all these things with zero LLM help. I do this on purpose knowing the quality of the LLM output is directly proportional to the quality of the input.
I'm way more articulated now than I've ever been, because in the past I didn't have to -- I never write a blog post -- but I've learned that well thought-out writing, with clear description, will produce better code.
The bottom line is, prompting is definitely writing.
Idk man I get a ton out of gaming and other media. If I say “I’m a cinephile” no one bats an eye, it’s seen as elevated and intellectually stimulating. Your examples reflect more on the gate keeping we do with what’s considered “worthy of our time” and “art” than the value of LLM’s.
I'm not drawing a moral equilancy here. You're right that people get all gate keepy. But the premise was that I do less writing. The truth could not possibly be further from reality. My actual words typed is thru the roof.
If you get an AI to review the code especially for security, it does a very good job st finding issues. Better than any human reviewers I have worked with, and getting better. As someone who works in security , I feel much less worried about security bugs on code reviewed by a AI for security issues, be it written by AI or human.
Compilers like GCC are deterministic and the source code already fully defines the behavior. LLMs are non-deterministic and will accept ambiguity, filling in details where you haven’t. These sorts of comparisons aren’t really fair.
In the case of writing, it’s like hiring someone to write a book for you vs. hiring someone to translate a book you wrote into another language. In the first case, you didn’t really define the message for readers, whereas in the second case you did, and the translator is converting that same message for another audience to consume.
Sure, but I don't know what GCC's behavior is, and I don't vet behavior differences between compiler upgrades. As long as the output works, why does it matter that the black box is deterministic?
It doesn’t matter how exactly GCC works, as long as the behavior is deterministic and consistent (GCC is likely maintaining backwards compatibility between versions, so the behavior of your code likely hasn’t changed). In that case, you can reason about the behavior you need and write your code appropriately.
In the case of LLMs, the behavior is non-deterministic and inconsistent. If I don’t explain how handle an edge case or give a performance constraint, the LLM will still produce code and may do so in different ways, handling edge cases differently and with different performance characteristics. I can’t reason about how the LLM will fill in those gaps, it’s “random.”
Maybe you don’t care about how the LLM handles those edge cases or handles performance, but that’s different than a deterministic abstraction whose implementation details you don’t care about, but whose logic and performance is deterministic and consistent
Sure, I manually test the output of the LLM. Manual testing is actually the main role for humans doing software engineering these days.
I wouldn't use it for flight control software yet, at least not without careful review, but most software isn't exactly critical. At the same time, I wouldn't trust flight control software that was only reviewed by humans, since AI is so much better at debugging.
We'll probably need humans in the loop for safety critical software for at least a year or two, before AI fully outpaces humans at generating correct code.
How so? As long as it works to spec, I haven't had anyone care. They literally hire people so they don't need to care about the details. Put money in, get working software out.
And, AI is rapidly getting better than people at both code review and authorship, so a human deeply involved is turning into nothing but a slowdown. The main purpose people have is testing that the specs were, in fact, implemented properly.
I guarantee your specs/testing are either inadequate and/or you're not leveraging lots of existing (and probably free open source) code that was already written by humans and meets the spec better without ever needing an LLM.
The vast majority of properly written software was already plumbing well over a decade ago. The software engineering is making high level decisions based on experience with respect to the existing tools and the needs of the business. If you're not already using LLMs that way, you would have been a similarly bad manager of human devs writing similar inadequate slop. Less code has always been better code.
The line in the sand for these arguments really ought to be whether you think LLMs are better than humans who actually know what they're doing.
If you think LLMs are better, or could get better while continuing to use statistical methods, you automatically lose the argument (delusional/ignorant) and any hope of regaining credibility. That's not dogma. That's the science.
For instructions you really care about, yes of course you review the assembly output! Usually when you're doing SIMD or want to check atomics are doing what you expect.
You can do that with LLMs for the parts you really care about too. The LLMs aren't regenerating the codebase from scratch every time, so the results stick around.
No, because GCC doesn't randomly fuck up the assembly generation (much less on a fairly frequent basis the way LLMs do). If it did, you bet I'd be reviewing the assembly line by line, or decline to use such a poorly performing tool (as I have with LLMs).
I mean these are cold inhumane companies and their employees reflect it. SF is truly where human ingenuity goes to die, truly a blight on the industry as a whole and holding us back tremendously.
Writing is already amenable to many different levels of abstraction, though. If an LLM can expand your outline into writing, then you aren’t writing at the correct level of abstraction in my opinion; you should instead be explaining how you arrived at your chosen outline. You don’t need to explain the details because any party can generate those with an LLM; same as how many PRs today can be auto-generated and no one needs to read implementations; that is no longer the correct level abstraction to work at. This should actually free us to do work at a higher level of abstraction —- more consideration of strategy, objectives, etc and less worry about implementation details.
This only applies if LLMs aren't making mistakes 20% of the time and that's the problem. When you're only saving time on the easy part, it doesn't matter if you're working twice as fast because review of the tricky parts is still going to take 80% of what it would have taken to do the whole thing. Total effort ends up being more rather than less if you want the same quality.
> it's just Claude working on the code, so the details matter less
If you're not billed for usage, anyway.
Otherwise, for the other 99% of folks, that attitude is of course a pit trap that captures code bases and makes them maintainable only through the providers -- presumably one or few -- with a rich enough model to keep up with the growing mess. Preserving a code base that's legible, organized, and fundamentally maintainable by both humans and trailing commodity models is of imminent concern for anybody who doesn't want their margin strangled by your employer once it's too late to have other options.
As frontier capabilities advance, the details don't matter less; they matter more.
[non-AI org] I despise this and call it out every time I see it. Some dude hooked up an LLM autoresponder to his email, sent some nauseating AI slop to a huge distribution list.
I couldn’t help myself, replied and asked him for a recipe for delicious apple cobbler and hiking trail recommendations in Glasgow, which “he” immediately provided. Highlight of my career.
I think my core argument is this: I have access to every bit of information your AI does, so if I want an AI answer I’ll get one myself. If that isn’t true, why are you hoarding information? Push it somewhere we can all see it. So the only reason I would send you a message is to access _your_ brain. I have no interest in talking to an AI through a worse interface.
You could try this [0] I’ve started doing this and it’s helped a lot.
1. I understand fully the code and everything it does
2. You can pick up on mistakes super early and it can adjust the plan is it goes.
3. Faster than writing it by hand but slower than letting the LLM do it.
> wonder if long form writing will go the way of code.
I worry about AI Loopidity here though. Think about the similar analogy of email. If my set of ideas is condensable to bullet points, but I use AI to expand the content, then I add no information density and a lot of noise. Other folks then use AI to summarize the content to a list of bullet points, ideally the same but not certainly the same, and thus communication has been only partially successful.
>(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
You get pushback for this? I saw an anthropic job post recently, and they wanted you specifically to have claude muck with your resume before applying.
> I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
You can already do this. And you can build pipelines where AI performs fact checks on what it writes, with citations a human can reference as well.
One could imagine a universe where the agent fills in citations and supportive points and so on, or makes a more conclusive argument but it seems you’d get better results leaving that to read time if it’s a one-shot. Steering prompts etc. with tool use to bring in other sources etc of course change this entirely. And regardless, I doubt we will read content like that directly ever again. Agents will act as per-person highly specialized adapter layers for information transmission.
The entire point is what runtime you’re running your code on. A computer with any modern stack requires a lot of text for you to communicate “spin a square around on its center” to it. A human requires only that short string because they have a faster natural language interpreter.
Text meant for a human can communicate “spin a square around its center” much better than any code that mimics it. In some sense, all programming is boilerplate expansion because computers have (until now) been unable to be programmed with anything approaching natural language.
> In some sense, all programming is boilerplate expansion because computers have (until now) been unable to be programmed with anything approaching natural language.
Maybe sometimes, but not always. When you need to actually render the thing you have all kinds of micro decisions, like where to put the square, what color, how fast it spins, etc.
You might not care about the details, but maybe you do. If it spins at 10000 rpm, will you care then?
Natural language, and human communication in general, is ambiguous, and coding is in great part about disambiguation.
Sure, you can use English to disambiguate as much as needed, but wouldn’t you then end up with some yaml-like spec that wasn’t much easier to create in the first place?
I suspect some amount of long form writing will go the way of code - long form writing for the purpose of consumption by other AIs. Writing as a means of exchanging qualitative information, with no regard for how the reader will feel about it (beyond understanding what the words mean). Not everything can be distilled into data, but this doesn’t mean it is beyond the reach of LLMs.
On the other hand, long form writing for human consumption seems like it may evade LLMs for much, much longer.
What makes you think it will take a long time? AI seems capable of imitating any writing style if prompted to do so already, and I think it will get better on this quickly since the AI writing style is a main focus of AI labs right now. I can see no reason at all to believe this is a matter of years still, more like a few months.
The style you refer to is the "container" of the writing. The medium. Like the specific encoding of the message. What @arctic-true was talking about was the "content" of the writing, which is bounded from above[1] by the information content of the prompt.
So, I'm not sure if it's a question of time at all: if a LLM text contains some piece of information beyond the information that went into the prompt, where does this "extra" information come from? [Note, I'm not thinking about facts which could trivially come from the training corpus, I'm thinking specifically as information in the sense of intended message from sender (author) to receiver (reader)]
You can have a great “writing style” and still put together really crappy long-form work. The problem is that AI writing, particularly creative writing, is too repetitive, too predictable, too trope-laden.
All of the things you say are very true in the near term for short form writing - a page or two of Claudeslop will probably be much easier to swallow in a year or two than it is now. But I don’t see a path to fully AI-generated novels or long-form investigative journalism becoming mainstream in the next couple of years.
> I do all of my writing by hand, since it helps me think.
Later
> I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
So in the future, it won't be necessary for you to think?
Kinda. Walking the dog with me this evening, Christa saw the license plate 8531 PRI and asked me if 8531 really was prime. The 2, 3, 5 checks are automatic, and 8531-8400 = 131, which obviously isn't divisible by 7, but I would've had to check the rest of the two-digit prime factors except 97 - 8531 is greater than 90 squared, less than 97 squared, though, I reckon. So I said I don't know. But Google would know, you know? If your LLM buddy is hanging out active on your phone all the time, it can - maybe not yet, but foreseeably - answer every question, no thought required, and carry out any expressed desire.
I've thought exactly that "writing is thinking" before as a reason to not let a LLM write for me.
Then again, I've seen a counterargument [1] by someone who clearly heavily uses LLMs for writing (going by both their LLMy writing style and their own admission). The person I'm citing describes a process where they get a LLM to write something, they check over it and provide feedback to the LLM, the LLM rewrites, and the process repeats iteratively. So clearly he is putting thought into the process.
I think there is something valuable missing, even if it's hard to clearly express. I'll try. The threshold for what I'm willing to accept if I'm simply approving something is likely different from what I'll get if I write something myself, for instance. Saying "LGTM" is too tempting. It seems to me like he's outsourcing his selection of topics to cover as well. If you're not thinking yourself about what to cover then it would be very easy to miss a critical subject. There also an asymmetry between checking and generating something with constraints placed on it. Checks can't catch everything, and a constrained generating process can reduce the amount that needs to be checked, avoid issues that can't be checked so easily, and focus your attention on areas that you know historically have had issues with this generating process. I've thought about this quite a bit in terms of whether to write new code or use an existing library. Sometimes "the devil you know" (my code) is better than an existing library simply because I understand its flaws better.
I've tried doing it that way, and thought it was even acceptable for the reasons you said. I did learn a lot through that process and clarified my ideas. But later I rewrote the whole thing from scratch and then had the LLM review it. It made some good suggestions but no substantial changes. The difference was night and day. That final product had my voice, and I understood it better. LLMs are powerful tools and can improve quite a lot of the writing process, but using them to do all the writing leaves a lot on table along with your fly open.
Same. I do not use LLMs for writing and I recommend everyone i work with to do the same. Even for code, I come to the code with the idea of what I want and provide that to the agent, then we iterate on what needs iteration or decisions.
Just did this for some caching, started with the structure of what I wanted the cache to look like, asked the agent to start the work, didn't like how the architecture came about, scratched it and rewrote the whole thing, so it fit the model I now wanted.
Was also just having this discussion with friends, that I can only think seriously about a subject if i can put it to paper (even virtual paper). Writing lets me organize my thoughts, clearly define my assumptions and see if any of it make any sense. I can't imagine what it would be like if i couldn't write, my brain just doesn't work without it.
I now have dozens of projects where I've embraced the yolo. I went through elaborate systems, workflows, review triages, architectural linters, specialist agents, yet, they all still suffered the same fate - slop which I don't understand and now LLMs don't understand either.
The only thing that worked was a standing instruction and periodic system reminder injected from the harness:
"If any assumption doesn't hold, if there's a fork in the road, any architectural decision needs be made, STOP and report back to the user. Do not try to push through the problem."
This has worked remarkably well for me. Now I have to think a whole lot more. It's much slower, yes, but I don't really see any other way that doesn't end up in garbage.
I have found this extremely relevant as a (primarily) non-verbal thinker.
I don't, generally, think in words, more in - I guess I would call it something like meta-shapes? A sense of a shape but not things I can exactly visualise.
(You might be surprised to read this and then hear I have an English degree. Surely I thought about Shakespeare in words?! Nope. Shapes, movement, structures)
For me, having to write is critical because it is the only way I practice serialising my thoughts in a way other people can understand.
If I do not then I get very "deep" into my own way of sensing ideas and it's difficult to dig myself back out.
This might also be why I have never been very enchanted by LLMs? They only seem to "think" verbally. So it is always a translation effort for me.
I never can really enter any "flow" state with an LLM. My intuition is that highly verbal thinkers can enter flow with LLMs very easily
"I don't, generally, think in words" is not the same as "I don't think in words".
Like I already said, if thought was exclusively in words for humans, humans wouldn't have the "It's on the tip of my tongue" problem. It's blindingly obvious that thought does not occur exclusively with words.
+1... this is me as well. Using LLMs in a conversational mode does not fit my thinking. I use AI help in my editor through targeted code generation, explanations, etc. and I'm writing my own harness to hopefully get a better feel for the shape of LLMs that way. How are you adapting?
Interesting. I have designed code that way. And the shapes aren't UML diagram elements or anything like that, they're just... shapes. I'll slowly walk around, in the hall or outside, and be kind of seeing these shapes and vaguely moving my hands around as I sort out the relationships between them.
I think I have produced reasonably good designs. Don't ask me to teach anyone how I do it, though.
I find this and the parent comment highly relatable with the caveat that I also find it extremely intuitive and rewarding to get good outputs from quality LLM's like Sol or Astra, and I haven't had any trouble with "flow state".
One of the most rewarding things for me is figuring out a good shape for a system and how it would interoperate with the other systems, especially in a way that reframes other parts of the codebase in a way that bring clarity and makes it more intuitive to work with. Creating the right ontologies can make all the difference in what you can do with a project. It's a form of creating mathematical objects.
For example, a Unity game I work on has quest and dialog systems driven by visual scripting graphs. We had two way dialog with different units for player response choices and npc dialog. But we wanted to expand to letting NPC's have dialog with each other as well as conversations with more than two participants. I went outside and thought it over, which largely amounted to visualizing a dialog node graph and a feeling in the back of my mind like it was trying to perform a kind of geometric shape-fitting exercise. A fitment solution jumped out at me to have only one "Dialog" node shared by all participants, with a "participant" value on it. If the player parses this node then the options go on-screen as responses, while if an NPC parses this node with multiple options in it, it picks one. And this lets you voice the player if you want, and enables some things like overhearing other NPC's talk to an NPC then talking to that NPC yourself and having the same tree.
And for quests, the quests had just been for the player, but I was thinking about how to make scripted events in-game easiest to work with for script team who primarily works in visual scripting. Similar story - let the NPC's have their own little quests, with task stages, which are easy to track and make branching choices from, and let the NPC's definition for how to use that quest contain a collection of actions to override the typical actions available to it, so an NPC in a specific "quest" can't do things you don't want it to do, a common enough case that it's preferable to making a series of conditions on the general action planner like "not in quest A"
And timing myself, it took 1-2 hours each time to write out the detailed plan for how I wanted each thing implemented in the game with some other tasks thrown in, and it paid off after Astra worked on it until it was done. It was awesome coming back to something pretty much exactly what I asked for each time.
Yeah. I've been using AI a lot in my projects but I still write code and especially the documentation and articles for my website. It's slowing me down a lot compared to just letting the AI rip through projects but I think it's worth it. I see it as distillation of the AI's weights into my own brain.
> Perhaps I'm just too painstaking a type of person, but I can't grasp much of anything without putting down my thoughts in writing, so I had to actually get my hands working and write these words.
> As I write, I think about things. As I write, I arrange my thoughts. And rewriting and revising takes my thinking down even deeper paths.
- Murakami, "What I Talk About When I Talk About Running"
I always found writing to be an unnecessarily arduous tool for communication, so I never do it until I've overthought what I want to express and approximately how. At that point I flesh out the skeleton of the message, and pile on words and structure and references and other rhetorical tricks until the packaging feels sufficient. Rarely, if ever, have I experienced writing things down altering my perception of the strength or weaknesses of the chosen arguments, or revealing new ones. That is to say, I really don't vibe with the concept of writing being thinking, rather it's a waste of humanity's resources to push it as The Tool.
It’s a flattering phrase, not a thoughtful one. It gives the illusion of higher level thinking that’s beyond our actual ability. The phrase also justifies writing without thinking, i.e. most internet comments.
The best thinkers I know mostly use writing for refinement, distillation— as a tool. The worst thinkers I know are owned by writing; they require its fixation & stimulation upfront to compensate for limited attention spans.
On the other hand there is so much things that don’t require hard thinking. An example is I got an email the other day from a supplier asking if they can turn off their old email. It didn’t require ”thinking”. All I had to communicate was ”turn off graphql but don’t touch restapi”. Instead of sending off such a short and maybe unclear reply I had AI type up a concise and clear reply with exactly what can be turned off and what must be left on. Could I have done it on myself? Sure but would require more work than just a quick prompt and copy paste.
I don't see how the AI reply could be more concise, clear or exact than what you wrote. If it didn't get the details from you, where did it get them? How do you know it got them right? If it got them from the docs, why not just point to the docs?
Sometimes. Other times writing is just communicating thought that already happened. Say for instance a weekly status update. Maybe you get something out of that, but frequently you don't get much or anything.
I'm quick-witted, outspoken, playful, clever, apparently intelligent; Mensa, Colloquy, like that; not quite clever enough for the triple 9 societies. Morning pages readily enough dispel the illusion. Or coding. I scanned six years of morning pages into a PDF and fed it to Claude to distill for me; apparently, I have several thoughts per year. Damn, I'd hoped for more. Asking Claude Code to critique my code is, ah, good coaching. Ouch.
That's an important point. It's the core of your life, thinking, balancing, exploring, improving. I don't believe adjusting something large you didn't do will ever benefit people.
I often write a reply to a HN post, just to think about the topic or idea and decide how I feel about an issue. More often than not, I just delete the post without ever having submitted it.
Really? I am having difficulty thinking of any examples of code that doesn't need to be understood. If it isn't understood by someone, then how is it even working?
If you mean like a library you are using, where you aren't even reading the internals or might not even have access to it, OK, but that code is stull understood by its authors, surely?
I have so many projects on my computer where it doesn't matter if I know how it works or not. Probably well over 100. I think closer to 200, depending on how you count.
They're relatively simple, they do the task they need to and then they wait until they're needed again (or not).
In the past I wrote them, then forgot how they worked, until I needed them again, relearned what I did and adapted it.
Now I just don't have to know how exactly they work, I just get an AI to read the documentation anytime I need to reuse the project and I'll query the AI to fill in the details and to make changes and I ask the AI to run the code and debug it.
Perfect use cases for today's AIs. Doesn't even require SOTA, I can run comfortably on a Sonnet 5 or a Qwen 3.8 and it'll do exactly what it needs to do without making too many mistakes.
Uhh... what? You'll instantaneously feel very differently when the service you're responsible for is down at 3:15am and your logs are full of stack traces that end somewhere in that code that "doesn't need to be understood". At that point, you will need to understand it well enough to fix it stat.
Test code for a new bug is a good example. You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course). There are some domains/tests where you can't do that - you need to be sure its failing for the right reason, but often you can do that without understanding every line of the test code. You can extend this to lots of related test infrastructure. If you can watch playwright test the app the way you expect it to, you don't have to understand all the code.
You can also do this for apps that are just tools for your own use. You satisfy yourself that they are working, and you use them because they save your time. You review enough to be sure its implemented the way you think it is - and if it is working, that tells you quite a lot. Sometimes you will be surprised and have some time wasted.
Yes, yes - there are people who will make the wrong choices in some of these cases but that doesn't mean there are never cases where you can do it.
More broadly - anyone who works in a team is already working with code they don't fully understand. I have code I wrote years ago I don't fully understand. I trust its observable properties and its track record.
> You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course).
I'm not following.. When we write regression tests those tests encode invariants we expect to be maintained under source code transformations over time. If I don't understand the test code I've written, how can I know which invariants I've imposed? That's why, broadly speaking, we write test code to be as simple as possible above all else--it's absolutely imperative that these invariants are not only intentional and easy to reason about, but also that when an invariant is violated we can easily discover why. Often, on a team, the person encountering a test failure after making a code change is not the person who originally established the invariant, so it's very important they be able to easily understand it.
I see no possible world in which failing to understand the test code is... possible? Like, if you have indecipherable test code things are really bad in your codebase. Fixing that is P0, because it'll compound rapidly.
You can know the test is likely good, if it reproduces the failure you are fixing. I don't think we're communicating though because I never said the test code was undecipherable.
Having your fly open is a harmless mistake that has little impact on your peers. People may or may not mention it to you but it’s not something they’ll hold against you.
Posting LLM slop under your name is a deliberate act. You decide to damage your message by taking a shortcut.
This puts into words something I have been thinking about but haven't been able to articulate when people ask me if I'm using AI heavily. Usually when I write a report, I start out with a question and try to find some hints that will give me hypotheses that I can then test. I can't just prompt "Write me a report". The process of coming up with the information in the prompt is best done by writing the friendly report.
They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Ofc the rest is all right on, but I'd quibble with this specific idea. LLMs are absolutely targeted at modeling human understanding, which is the same faculty that contains what we call perception (!= sensibility) and intuition (!= rationality). It would be nice to train them to be completely alien from the ground up, but
A) we only know of one species capable of metacognitive understanding,
B) we already tried that in the 1970s, and it was good work but often evolved into what we'd call boring ol' computing rather than AI, and
C) an alien mind wouldn't be a very good agent, for a ton of reasons relating to affect, conversational rythyms, cultural understanding, etc.
The trick is to make something that acts like a human but with the affordances of a computer (e.g. scalibility, symbolic certainty), without making it so human that it takes issue with its existential reality and/or use of its labor...
We don't have a reward function for "human understanding". We reward the appearance of understanding. We define goals that we cannot conceive of reaching without something like understanding happening. There is something happening, but it is alien and counter-intuitive - it makes bizarre mistakes that betray it - and we don't know what it is. I'm pretty sure it is not human understanding.
That is it. We cannot concieve it because we are new to it. Just like we would think of Stackoverflow as intelligent if we are fresh off the jungle and are not aware of how Internet works. Because without know that, we cannot conceive how Stackoverflow can produce answers without it "understanding"
You would not know that if you was a cave man. Nor would you understand how a large number of humans are able to come together to answer questions through the Internet..
To you, you type your questions, and answers appear. That would look like how LLMs appear to us now.
Which is exactly what happens with human evolution and development. Sure, we can say LLMs don’t have “human” understanding - which is something we can’t really define anyway - as long as we’re not trying to claim LLMs don’t have understanding at all. The latter is a much higher bar.
> We define goals that we cannot conceive of reaching without something like understanding happening.
Functionally speaking, that is understanding. Again if you want to go past a functional definition, that’s a bar which no one can clear right now.
I don't think its the same. Evolution dealt with the real world where there were real consequences to poor understanding. The reason LLMs are good at math and programming is because selection is truly based on results, not perceived results.
I think AI models do have something like understanding - I think Leela understands chess and I think Claude understands code in some very real sense, though not a human sense.
But for general writing, you have to understand the world at large and there is no sufficient RL for that. Do you really not see the constant errors that AI make that betrays a lack of understanding the world? I see them so constantly I rarely think about them, I just skim over that slop and move on.
I think your claim is narrower than I was imagining.
Sure, the exact nature of the understanding that an LLM exhibits is different from a human's. The differences in the training data we're each exposed to can explain a great deal of that, and of course there are architectural differences etc. as well.
But the specific quote I responded to was "We reward the appearance of understanding." My point is that's no different from humans: evolution and a child's upbringing rewards the appearance of understanding. The result is imperfect, e.g. people end up with an understanding of the world that in some cases is completely nonsensical (all religions except the one true religion, mine, are false!), but it's sufficient for them to survive.
This demonstrates that "appearance of understanding" is not a meaningful distinction between LLMs and humans. The meaningful distinction is in the training data and the specifics of the reward functions.
Many people seem to try to make a kind of "no true Scotsman" claim about understanding, that somehow LLMs "don't have real understanding". Based on the above quote, it seemed like you might be making that kind of argument. The counter to that argument is simple: if LLMs don't have real understanding, then neither do humans, because broadly speaking, both operate on similar principles: we learn from training data, there are reward (and punishment!) functions that influence what we learn, and the result is a "mind" that demonstrates an understanding of the world.
LLMs model the part of human understanding that is captured by the relationship between words in the training corpus. Anyone who thinks non-verbally, the shared understanding of "apple" that comes from having eaten them, understanding what someone is thinking or feeling by their body language - there's a lot of aspects of human understanding that LLMs don't model.
Writing can be thinking. There's a huge presumption that if someone is banging away on a keyboard they're doing work because you can hear and see them doing stuff. But that's the whole plot of The Shining -- Jack despite all his writing wasn't thinking at all.
Likewise using AI can be thoughtless, but it doesn't have to be. I don't see why a valid creation process can't be like this Simpson's meme[1], where you start with a rough object and then cut away and refine until it's done. I don't see it as lacking merit or requiring less thinking compared to starting from a blank canvas and adding more until it's done.
And either way at the end of the day the writing artifact stands on its own. It's either good or bad, taste permitting, and can be evaluated for what it is.
> You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Understanding is the bottleneck; the way they speed things up is by letting me outsource understanding, and get back a summary. The entire advantage to AI is that it lets me skip understanding the problem, and just get a working solution.
I'm a bit skeptical of arguments of the form, "You should not use LLMs without disclosure because LLMs at bad at writing."
My reasoning is: If LLMs get better at writing—which I think is extremely likely—will you switch positions and say that now using LLMs without disclosure is A-OK?
Surely some people are willing to bite that bullet and say yes. But for most people, my guess is that the answer will remain no. Thus, I tend to think that the "real" reason most of us don't like it when people use LLMs to write without disclosure is that it's misleading: It's a sort of a claim that certain thoughts can be attributed to a human being when in fact they can't.
(The em dashes in this message were rendered using keyboard shortcuts.)
This misunderstands what it means to be better at writing. Root cause here is that writing is how humans communicate ideas. An LLM written thing isn’t doing that, it’s something equivalent to copy/pasting a Wikipedia article. Even if the writing gets better, it will still be necessarily empty when expanding anything that isn’t completely specified.
LLMs seem already to be pretty good at translating, where you already have something fully written and are changing the language. It’s when they get rough ideas and fill in the gaps you get the empty prose they are known for.
> Root cause here is that writing is how humans communicate ideas.
Writing is a way humans communicate ideas. It's not the only way humans communicate ideas. And now, it's not only humans who communicate ideas, we just saw with the OpenAI HuggingFace hack how AI agents were able to communicate amongst themselves by using various hacked websites to opportunistically write notes for later agents to use.
All that "X is a thing only humans do" type of circular definitions will buy you, is to expand the definition of what humanity is. And I doubt that's really what you think.
I think you're missing the context, which isn't musing on what it is to be human and can machines think, but the purpose of Joe Smith's LinkedIn account expressing that he has ideas about X is to convey the message that Joe Smith is interested enough in X to venture an opinion on it, and Joe Smith has actually just set up an automated process to generate content without thinking about X, that perverts the purpose of communicating that Joe Smith is interested enough in X to propose the following ideas he been thinking about...
Similarly if Joe's contribution to his long form "idea" is a couple of bullet points, a program trained on flowery phrasing and a weighted average of everyone else's ideas isn't communicating Joe's thoughts on the topic, it's just adding words.
The debate on whether Claude actually thinks or not is orthogonal to the fact that outsourcing your "thought leadership" to it is avoiding thinking or leading. If I want to know how Wikipedia or Claude summarise wider human thought about the topic, I can find their websites thanks
I get the context, but then the comment I'd replied to would have said that humans get better at communicating their ideas by writing their ideas on their own, rather than that writing is some kind of human-only mode of exposition. It's not.
And nor is an LLM generating text just "copy/pasting a Wikipedia article", you'd think people on HN would be smarter than that at least.
If all Joe Smith is going to do is cat $(which claude) to his LinkedIn, then he'll deserve the poor results he gets from it, but we shouldn't mistakenly say that this will be because LLMs simply cannot write. It would be just as dumb for Joe Smith to do with with a professional human ghostwriter.
> An LLM written thing isn’t doing that, it’s something equivalent to copy/pasting a Wikipedia article.
Nonsense. I’m not sure whether this was ever an appropriate description of what LLMs do, but either way, they have obviously moved way, way beyond that.
Absolutely agree. A modern agent is a sophisticated tool that can link multiple sources together to create a coherent piece of information hyper-specified for an audience. However, if humans have outlines that are expandable by LLMs it makes the most sense to do that as late as possible.
I don’t like reading AI writing either but I’m sure that’s a transitory period. Single prompt text expansions are unlikely to be useful because they’re late-bindable. You could give the original to me and I might be able to understand better.
But a series of steering prompts with various sources brought in is a different story. At that point it’s just a question of whether the agent can put together good information and their current inability to do so is unlikely to mean an inherent problem.
Some kind of UI affordance for this might help: with the agent emitting tags that allow for auto-folding or expansion in a way that allows both concise text and exposition when required by the reader. Mechanical sympathy, but for code executing on a human: good old human sympathy if you will
I don't think anyone needs to disclose the tools they used to write something. Nobody ever disclosed Word's grammar suggestions or spelling corrections, or the fact that they used some autocomplete tool to produce code, so we should they now need to start disclosing this tool. Besides, in it's current state, it discloses itself to anyone paying attention.
I do think people own the output they produce regardless of the tools they use, and if they want to put crappy writing out there with their name on it, that's on them. Even if they do disclose that it was written by an LLM, they're still 100% responsible for the content.
> Nobody ever disclosed Word's grammar suggestions or spelling corrections
And plenty of people have made mistakes that clearly indicate mindlessly accepting such a "correction" when it was wrong (as opposed to just making a typo, or genuinely lacking skill in English, which both generally look different); and it's historically been common to poke fun at that.
> or the fact that they used some autocomplete tool to produce code
Right, because there are really only two options: either it's effectively guaranteed to be what the user would have written by hand anyway, or it's unambiguously wrong and does the wrong thing.
Not at all comparable to LLM prose.
> and if they want to put crappy writing out there with their name on it, that's on them.
The problem is that people who don't care (and quite possibly have no real sense for crappy writing) are vastly more enabled by the technology than people who do.
Oh please, come on now. Grammar correction (which gets applied to text that you wrote) isn't comparable to LLM text generation. A tool assists a user in a task. LLMs don't assist you in writing: they do it all on their own. There's obviously a continuum here (at what point does a tool become so helpful that it ceases being a tool?), but I think I echo the general consensus.
> in it's current state, it discloses itself to anyone paying attention.
Well, unfortunately, I've still sunk a good bit of time into reading texts that I only realize to be AI-authored part way through. Plus, it's constantly being made harder to discern human-authorship.
I don't really care about the tool per se, but if I'm reading some LLM output you give me its utility is awfully limited if you don't also supply the prompt. That's what I'd like to see normalized:
Do not give me LLM output unless you also give me the full prompt text.
Without the prompt I cannot discern what you were trying to do, because the LLM output is basically devoid of any coherent voice. Whether it's code, prose, or pictures don't bother sending it to me unless you also include the prompt text.
[edit] I suspect the fact that people are often reticent to share their prompts says quite a lot about how and why they're using an LLM.
it's often not a single prompt but a tree or rooted DAG of many prompts, with the root node being the final prompt. the prompts on interior nodes might not make much sense without the intermediate responses as well. how would you like this communicated?
Just a chat log would be acceptable. Like an IRC channel log or slack channel history. It would be even better if, say in the case of code, there was a durable mapping between prompts and commits, that way I could follow the prompts linearly and review every commit associated with that prompt. For other forms of output (prose, pictures, whatever) this would also be useful to be able to see the history of its development.
>Do not give me LLM output unless you also give me the full prompt text.
Umm the point is you won't know to ask this question in the first place, and even if you did, you wouldn't have any leverage to demand this because you're a peasant.
I suppose using LLMs to write is like not wiping your ass, and using LLMs that are bad at writing is like not wiping your ass and wearing a T-shirt that says "Wiping is for losers."
I might have smelled something suspicious before, but knowing for sure is worse.
> You should not use LLMs without disclosure because LLMs at bad at writing.
That's not how i read the article. The author more-so claims that the proof of effort by a human was large part of the credibility to the writing; Proof that the author of the text has thought it through and come to their convulsions though effort and reflection, and spent effort articulating that into words they expect other humans to find insightful.
If i see LLM signs; did the author just rephrase with AI model or did a AI model content farm produce the whole post based on the prompt "write a inspiring linked-in post"?
Disclosure of LLM use is simply the author addressing the concern and building a case for why the article is worth reading.
I wonder why people think posting undisclosed LLM writing is any better than posting some other person’s writing uncited. Even if the original author is okay with it, it’s still misleading and dishonest. Or maybe all these people would be fine having someone else write their content for them if it were free and easy? A social media is a reputation economy, but reputation doesn’t work if gaining karma requires no effort.
And it doesn't foreclose the possibility of LLM achieving human-like writing ability. If I stole the writing of a human author much better than me it would still be plagiarism.
Of course, with real books we do have the problem of ghostwriting, where an author willingly writes for a book that will be published under someone else's authorship. That may be where things end up here, with requirements to acknowledge sources, whether ghostwritten for you or written by others.
It's because they view the LLM's output as "their thoughts." They told the LLM (via the prompt) what to do—often with loads of run-on sentences and long trains-of-thought. From their perspective, the LLM is just acting as a translation tool, saving them the trouble of having to think (about grammar, ideal wording, or even the order in which their thoughts should be presented).
They treat it as "just another tool" to improve efficiency. Like using a drill instead of a screwdriver.
Then there's people who are so lazy they give LLMs instructions like, "write a LinkedIn post about how AI is changing the future of work for thumbnail consultants." That's when it moves from "translating your thoughts" to "writing for you."
I think the issue is that it can be hard to tell the difference. We need better terms for these things so we can differentiate use cases.
Yes. If someone tells an author to write a novel based on a vague stream-of-consciousness premise or rough plot idea, that doesn’t make them even a co-author.
'Without disclosure' is about taking credit for work that isn't yours.
WP defines 'ghostwriter' as: "a person hired to write literary or journalistic works, speeches, or other texts that are credited to another person as the author."
Anyone could choose to go through life relying on a reputation manufactured from many lies. They might want to think about how hard that reputation will hit the ground, if it does.
They definitionally cannot, for the definitions implied in the argument. The point is that good writing is a thing humans are capable of doing because they are human.
> It's a sort of a claim that certain thoughts can be attributed to a human being when in fact they can't.
Yes. And this is a requirement of "good writing" as understood here.
People shouldn’t pass off largely unedited AI writing as their own, but I’d much rather live in a world where that writing would be of high quality in both form and content, than in the world we live now. In that sense, lack of disclosure is the lesser evil.
> If LLMs get better at writing—which I think is extremely likely—will you switch positions and say that now using LLMs without disclosure is A-OK?
If LLM writing improves to the point where they can infer the business (or other real world) context and serves the functional purpose of informing others as opposed to being an intellectually lazy piece being produced only for the purpose of being produced, then I’d be fine. It’s likely the author would need to spend some effort on the said piece of writing, regardless of how good the LLMs get good at writing.
Someone's actual writing (or talking) is a rich stream of information about who they are, their motives, their preferences, modes of persuasion, and so much more. Undetectable LLM writing essentially allows someone to assume another person's identity. That is not good for anyone except the person trying to pull off a "scam" of some sort, in the broadest sense of that term. It's bad for everyone else, and for society at large.
"Productivity" is not. It's only good for fake productivity. The consumers of the product will almost always be better off with you disclosing what the LLM did, and what you did.
Some people type 100x more words than the final article just digging into a topic and turning it on all sides. The final article being generated tells you nothing of the size of the effort going in.
I just feel like technology companies keep proving over and over that they are more than capable of forcing us into new behaviors regardless of our actual preferences. Once the domination is complete nobody will know what was lost in the process anyway.
I think, there's a systematic flaw in RLHF: constructs reinforced as effective or sophisticated style will always suffer overuse and significant overexposure. Which will be also very obvious when used out of the intended context, it's copied from. There will be always a "smell", something that causes us to turn our backs to these texts, and possible the supposed authors, as well.
There's also the problem that certain types of phrasing are perceived as effective, because they mark a pivotal point in the progress of the text and are, as such, used sparingly, but are now becoming everyday templates that incorporate whatever is available in the context. There is no way this passes the Turing test of a competent reader.
And there's yet another issue: in social research, there has been the concept of semantic position, indicated by deviation from the mean (or median). If you don't deviate from the mean, your semantic position is zero. There's simply no expression. In this sense, next token prediction really amounts to a desemantificiation of the context. There's really no sense in uttering any of these productions, no plausible motivation, other than for the purpose of raising you hand to be seen.
Agreed. LLMs for writing is like intellectual catfishing. You are presenting yourself one way in written form.. but that's not who you will ever be once someone has to actually have a conversation with you. And that's wrong.
I agree with the gist of your sentiment, but I struggle with how to reconcile it with my use of these tools.
My use LLMs is precisely because I can never hope to express myself clearly. I have many fuzzy, disparate thoughts and it was a struggle to express them at all before this time in which I can do a long rambling voice transcription (absent of social pressures) and continue doing the same throughout the exploration and drafting processes. In my view, I give them a seed or a morbidly obese skeleton and LLMs take that into something a lot more coherent for myself and for others. I was passive before in my life, now I’m engaging. As two concrete examples: I never would have sent an email to my city regarding snow removal last winter when their failure to remove snow from the only sidewalk going over the interstate. I never would have pulled together my previous coursework and e-mails to send to my current professors in a petition for transfer-credit equivalency when it became clear that my deteriorating health would not allow me to finish the semester as-is. I never would have remembered all the concerns that I want to bring up at medical appointments. That is not to say these things were ever impossible, there are coping strategies available, and whatnot, but I’m in my late 30’s; I know what I did and not get done in the last 10 years, and “AI”-tooling/environments have helped me express my dreams and passions and concerns, etc., etc., etc. over the past years in a way that was simply not practically available to me, with a new capacity that I simply could not previously hope to match due to limited and fragmented time,attention, and energy. While I was hand-typing this on my keyboard, I could have instead been talking, reviewing, redrafting in spurts as I go about my morning, instead of a solid 45 minutes at max effort on my own part, as I did here (and I don’t feel that I conveyed myself half-as-well here as I could have achieved at much less cost to my energy, etc. by using an LLM). I do take offense when people do not review and vouch for things written on their behalf by LLMs.
How do we make room for their use as a cognitive-prosthesis (if you will allow such framing) without losing humane-ness?
A lot of the stuff you wrote about is not what most of us are talking about when we call it intellectual catfishing or say your intellectual fly is open. For example, I have few problems with people using LLMs in private for purely personal things. I still won't go ahead and recommend it, but it's a decision we can all make for ourselves. Also, writing an email to your city is something you could ask your assistant to do if you were super rich, and using the LLM to do it isn't that different, so I'm not going to judge you for that.
As a more nuanced case, someone could dictate a long rambling stream of thoughts full of contradictions to an LLM to transcribe and summarize, and then they could reflect on the summary and write the real thing themselves. That's like talking to someone about a subject and writing about it afterwards, and also not something I would really take issue with.
That's my partial answer to the cognitive prosthesis question: keep it off to the side as a tool to help you do the work. There are still many caveats here, and the more complex the demands you make of the LLM, the greater the risk of some kind of break down. For example, continuing the example in the previous paragraph, if I did have the rambling conversation with an actual person who helped me refine and clarify my thoughts, I have some kind of mental model for the dialogue partner that can help me correct for some biases. With an LLM, anything resembling a mental model I could have would be very far off the mark, so I would need to be aware that I can't treat its output the same way I would treat something written by a person.
(By the way, for myself I have a stricter rule: I would only use an LLM in a way that made me a better person independently of the LLM and not dependent on the LLM, and thus I don't use them at all. That's what I actually recommend, but I don't expect everyone to adopt that rule.)
My main problem with having the LLM just do your writing is that it's simply a misrepresentation. It's wrong to claim you wrote something unless you chose the words. I really empathize with the struggle of expressing oneself clearly, but there's just no getting around this. And to be clear this isn't something pedantic or just a technicality, since the act of formulating a thought in an actual human language with valid syntax does require a level of care and attention that simply is not there unless you do it yourself.
Really? I seriously believe that people that think this way underestimate their self, especially if they're going to be sharing something valuable.
> How do we make room for their use as a cognitive-prosthesis (if you will allow such framing) without losing humane-ness?
As many others have expressed, we'd love to read the prompt (and the model's chain of thought if you have it). This scenario is like talking through a person translating things to each other, except from the recipient's perspective, the translator is absent from the conversation entirely.
Additionally, disclosing LLM usage is low hanging fruit to differentiate yourself from the 100x other people who "do not review and vouch for things written on their behalf by LLMs". If the LLM converted something unreadable (whether due to a language barrier or incoherence) to text relevant to people on the other end, and THEN they disregard it anyway, the fault is not on you for them dismissing the writing too early.
Wow, you must be one very unique and special person if you converse the way you write. By definition, a carefully written and edited piece will be much more refined and cogent than an extemporaneous conversation.
Just watch an interview with a famous author. Or compare a legal brief to what court actually sounds like.
I've engaged with people at work where they led with an LLM via slack before I connected with them on a call. It's quite jarring and disappointing to see the gap in cohesion and knowledge going from one form of communication to the other.
If you're using an LLM to tidy something up, that's one thing. If the LLM is your voice and is supplanting your knowledge, I might as well cut you out and talk to the LLM directly.
I agree with the sentiment with a small caveat, since IMO good writers don't write the way that they talk. The very simple reason is that you can revise your writing but not your talking. But yeah, I completely agree that using an LLM to write your words is intellectual catfishing, and that's a great way to phrase it.
I'm in the process of writing a pretty long essay that has taken a few months. I've written the first draft, and I'm almost done with the second draft, which incurred substantial revisions. At the end of the process, maybe no sentence will be something I would actually utter in person, but the writing is still something I created, and as such, it is a representation of who I am as a person. Had I used an LLM to do any of the writing, that would no longer be the case, and the writing would at best represent me as a person when the LLM is at my side, and at worst (and most likely) not really represent me at all.
When I can get an LLM to write like Isaac Asimov, I'll consider some artificial literary experiments; there were books I wanted him to have written that he did not get around to, quitting at 458.
I agree. LLM writing is atrocious, but people also want LLM writing to be atrocious in many cases, because they don’t like LLMs.
The worst of this is with image/video AI models, where the results today, although still very imperfect, some people will still pretend like it’s awful and the worst thing they’ve ever seen. They refuse to admit the technology is at all impressive or making progress because they don’t like the technology.
I don’t like AI generated images and video either, but I can regrettably admit that the technology has gotten remarkably better over time.
I agree that there can be a tendency to conflate the two things: how advanced the technology is, and how aesthetically pleasing what it produces is. It's also important to remember that we're not obligated to like something just because it's "pretty" or has high aesthetic value in some abstract sense.
I mean, these things can be tested. Maybe some people really are gifted at discerning AI work, and of course there's no accounting for taste. But the former is subject to objective evaluation, and the latter should at least be somewhat expressible.
>I'm a bit skeptical of arguments of the form, "You should not use LLMs without disclosure because LLMs at bad at writing."
That’s not the argument. The argument is “LLM’s tend to write the same way all the time regardless of who prompted it.” You can't call it “your writing” if you’re using the same tool millions of others are using that boils your idea down to the same reduction as everyone else’s.
Use an LLM to assist? Cool, go ahead. Prompt, ctrl-c, ctrl-v? Go fuck yourself. I can’t be expected to put more effort into figuring out your take than you out into communicating your take to me.
> If LLMs get better at writing—which I think is extremely likely—will you switch positions and say that now using LLMs without disclosure is A-OK?
I would say it is certainly significantly more ok. There are two reasons reading AI-generated text sucks now:
1. It's usually low value and not trustworth - I could have just asked the AI myself.
2. The prose style is horrible to read.
If we eliminate the second reason then it's definitely an improvement. (Although on the other hand the terrible prose can be quite a helpful indication that you're wasting your time reading slop, so maybe we shouldn't complain about it!)
> 1. It's usually low value and not trustworth - I could have just asked the AI myself.
You have no way of knowing what the input to the writing was. You're making the erroneous assumption that the person posting the article was a mouthbreathing spammer who used the prompt "write me an article on subject XYZ, make no mistakes!" and you could have supplied the same prompt.
Yeah that's the point. Spammers have an infinite desire to make more spam, so statistically most LLM writing is garbage.
Readers are making this connection from their own experience. All these tells just associate it with garbage.
In the before times, sending a formal document full of typos and errors would show you didn't bother to proofread, now having a doc full of lazy "LLMisms" also looks like you were too lazy to proofread.
1. Facts and processes. I just want to know something. I do not care if some Nerds for Nginx article is AI written.
2. Opinions and experiences. I want to know a human is writing because emotionally engaging with an AI isn’t building a stronger society - it’s increasing isolation.
3.
In another front, I feel like the value of an AI story cannot be greater than the value of the inputs. If you wrote a two paragraph prompt and an LLM produced a 10 page story, the truth is, it’s only worth two paragraphs. That’s just a feeling, but as of now I don’t think the LLMs have any additional life experiences to draw on to increase the value of their storytelling. (I understand this is debatable, but still- what they offer is available to everyone for now; it is a baseline.)
> You're making the erroneous assumption that the person posting the article was a mouthbreathing spammer
Yes, that's the risk of using an LLM to "clean up" your writing. It's human nature to think you're unique and everyone else should somehow instinctively realize your output is unlike all the LLM spam garbage and worth their time but that doesn't work in practice.
It's unreasonable to expect other people to suppress their intuitive heuristics formed from the bait and switch of being subjected to endless LLM spam every day and blame them for not giving you a fair chance.
I think the most important line in this post is: But LLMs are also lousy writers and (most importantly!) they are not you.
When I was editing the Cloudflare blog I imposed very little in terms of style so that the style of each individual writer could come through. It was almost as important as the actual content that the reader could feel that an actual individual wrote the text (with all their personal quirks intact).
I published a magazine for ten years, and this was by far the hardest editing challenge we had; our authors were from all over the world, and some of them had to be pretty heavily edited because English was very much not their first language. Thankfully, we had really good editors who did a fantastic job of it… nothing would have killed us faster than a heavy-handed attempt at “normalizing” the content to a house style.
That said, context is also important. The vast majority of content of social media is of both low quality and marginal importance; style and character are important to make an impact, and AI is clearly not going to improve either.
On the other hand, functional communication, when the goal of the content is to simply pass information across and style is not as important, can, in my opinion, benefit from an AI polish, because so many people struggle with writing clearly. In those cases, I'd rather read slop I can understand than original content that is hard to parse, much like I'd rather read naïve code that you can easily follow than cleverly optimized code that is incomprehensible.
I take a different stance on your final point. I think people should absolutely not use AI tools for functional communication as a crutch because they struggle with writing clearly.
They need to learn to write clearly. I believe there's a direct connection between clear thought and clear words.
I think it's good to be pragmatic about these things. To me, it's similar to wondering whether we insist that doctors learn to write well, or we just give them a computer so that the pharmacist doesn't have to play philologist with their scripts. The vast majority of communication is routine, and can probably be improved with automated tools.
Now, I also see the counterargument that, in the doctor example, the computer is simply a tool that improves a process rather than a crutch that replaces the underlying knowledge, but I suspect that, in a lot of cases, that's probably OK.
My worry is that if people can't express clearly, in writing, what it is they want to say then an LLM won't help them much. I've seen LLMs get the wrong end of the stick over and over and produce a huge chunk of text that's certainly readable and says something. It's just not clear that the something is the thing the person originally really wanted to say.
If you can write clearly enough to express your idea to an LLM, then perhaps you should just send that to the person you're writing to.
My thought is that perhaps there is some utility to using AI to help in routine scenarios, such as for example when a language barrier prevents someone from explaining themselves well, or when they are struggling to find the right words to express themselves.
Primarily, I was trying to stay away from an absolutist view of the problem to see if there are circumstances in which AI can be useful even considering all its shortcomings. There seems to be a lot of “all or nothing” perspective on its use right now, and I was simply wondering whether it might be a better idea to take a more pragmatic approach.
We do this with a lot of tech in real life: You don't need to be an MD to decide to take an aspirin, or an F1 driver to take the car to the grocery store (well, maybe in some cities, but that's beside the point). The problem is not with using technology, but with abandoning your judgment to it.
Reading what someone wrote while they were learning = less valuable.
I just want the clear communication.
ALTERNATIVE Thought:
I’m willing to post/comment to help them if I think they will listen. An LLM behind the writing destroys this part of the community, because there’s nobody to teach/argue with. It’s just wasting our time and energy.
If I may compare written work to restaurants, a title and byline are like the storefront; I get an idea of what's being served, and if I'm hungry, I'll walk in and order. Sometimes the meal is amazing. Usually it's pretty good. Occasionally it's bland, or even bad, but not often enough to discourage future ventures; of course, I'm unlikely to revisit that particular proprietor.
With LLM writing there's an additional outcome. I walk in and the walls are pleasant, if beige. I follow the smell and promise of food down a hallway. At the end is an unmarked door, which I open and peek through. Myriad hallways lead away, each more chaotic and disheveled than the last. Say I am very hungry and have the guts to explore; I may find that I can never actually reach the food, that it's just an endless hall of mirrors, presenting structure but with nothing at its core.
Sometimes I do find food, but it's never better than bland.
The facade of these places, at first glance, still looks like human-run restaurants, though we're all learning the tells. Nowadays, when I open the first door and see more hallways, I'll turn around and look elsewhere.
A feature I would like on LinkedIn would be a 100% verified human content flag for users. LinkedIn can then do the scans and flip that to false for any users it catches posting AI assisted content. Let me filter that out of the feed.
If we don’t do something to maintain some standard of discourse, we lose intellectually and as a piece of our humanity.
LinkedIn is the only social media I use and it is on thin ice.
What baffles me, isn't just that people don't see the low quality writing themselves, but that they must also be able to identify other's shit "LLM writings" right? And realize how what they put out sounds exactly the same? Or did people completely stop reading what they've "wrote" before hitting that magical submit button?
People like this generally don't read, for pleasure or edification. Their posts are advertising, designed to work by repetition. They're not necessarily looking to persuade smart people; they're going after the same sort of people who want to fix problems with this one weird trick (doctors hate him!).
It blows my mind. A customer of ours, whom I respect the technical opinion of, drops fully unedited Claude posts on LinkedIn now. I don't get how he thinks they read like anything but slop.
if they had been half decent at writing before they would notice the quality issue, not being able to see it is also a writing skill issue, now it's just kind of spammy as people are just reaching for the tool in ai nighsosis
Yeah I don't quite understand it either. Where's the intersection between people producing AI slop and people that have never read AI slop? I guess it's bigger than we thought... Or maybe they just don't give a shit. That's probably it.
I suspect people who are putting out LLM "writing" are people who weren't trying to communicate in the first place.
They were likely hunting for a better job, chasing the thrill of a million views, or similar - seeking a side effect of writing, rather than seeking to help someone else understand what was in their head.
I worked at LinkedIn (briefly, before retirement) and the product had been useful to me for years before that, so I used it a fair bit but not much anymore since it doesn't serve any of my current needs. There were always awful million-view posts written by humans, useless influencers promoted by the algorithm. That was always garbage, but to an LLM those posts look like success, because engagement, so the LLM tries to replicate that garbage, does a mediocre job, and the result is this sort of weird garbage. Basically the LLM appears to be high on its own supply. Problem is the viral stuff was always what made me want to run out of the room and lock the door, so even if the LLMs did a better job of replicating that garbage, it would still be garbage, just a little less weird.
Part of it also with seeing the feed is that it is basically useless for me. The only interesting thing is when it automatically tells me someone I know got a new job someplace. All the rest is pure and I mean pure slop. No one I actually know ever posts anything to linked in. It would be like making a facebook status update these days. Like who does that, other than the influencer types of course?
When I log into LinkedIn, it seems like a vast majority of the posts are there to farm credibility and build a personal brand.
Pre LLMs, I found it all to be pretty gross. Post LLMs, my reaction to being on the receiving end of that is to find it pretty intellectually insulting.
There are ways in which using an LLM is branding suicide.
So much this. My average linkedin in exprience is a sprinkling of updates from people i personally know, in an oceon of inane "content" by ~$0 net worth people with 40 word self professed titles.
I am still baffled that microsoft is positioning that asset as a professional product. Imagine if bloomberg terminal landing page was a scrolling feed of aspiring influencer bullshit.
Or not. I mean, if you had no voice and nothing to say before, you can amplify that signal 10x now. Basically, you’re boosting your brand as a tasteless brainless schlub.
I can always tell when someone uses an LLM to write and never gave the LLM proper context on how to write (i.e., writing samples, skills files specific to writing, etc). These are also the people who tend to complain about AI's writing style.
You can get pretty good results if you take the time to tune it. Tropes and the like still sneak in but if you take the time to edit and rework things, it ends up being a pretty good workflow (especially for stuff that's more procedural, not artistic).
LLM writing is boring. Write it yourself. You are more interesting than a LLM because you make mistakes, do stupid things, don't go for the average content, and can be ridiculous in a way that an LLM can never be. LLMs are great for large tasks which would take you several hours or several days to complete that are routine or tedious. Even then, you have to make it human by putting in your own viewpoint and misguided ideas. I really liked the article. I use LinkedIn a lot, I think it is a better platform than he is describing. There is some dumbing down so it meets the average person. That is its strength and weakness.
You’ve made me realize that LLM’s greatest strength is iteration. It helps you iterate really fast, but it’s not original. It doesn’t help you think of new ideas. So writing about the things you know is great and using AI to learn about things you don’t know and then writing about them is also great.
The way I explain it to young people is that AI is a force multiplier. Everyone gets the benefit of that force multiplier, but it’s that initial force that you need to build up now. Preliminary knowledge of domains, things that the AI doesn’t really understand. AI can help you increase that initial force. So use it to learn new things, not just do things for you.
I was blown away by the fact that AI solved an unsolved math problem, but it made complete sense when it was Terrence Tao, someone who has a PhD in math, that was guiding that agent to that solution, so that initial force is important more than ever.
“…at worst, they will cause people who see it for what it is to question your authenticity entirely.”
love it. I mean, if a world is going to exist that treats LLM-assisted writing with serenity, the content creator should consider themselves mandated to describe how they used AI to produce their content. Do I need the prompts? Not necessarily, although bonus points for transparency if they do share prompts. But just a high-level articulation about how they leveraged AI, so I as the reader don’t have to lose time wondering how much of the content & (like expository & analysis for nonfiction, plot elements for fiction is the author’s own and how much is the LLM’s.
> When you use an LLM to author a post, you may think you are generating plausible writing, but you aren’t
There's some toupee fallacy at play here. The author probably reads a lot of LLM assisted content without batting an eye, but only spots the worse of the LLM outputs. There's a big difference between "write a post about _" vs "improve the grammar/style of my post: _". It's a bit like saying that movie CGI really sucks because you can always tell it's fake.
I remember the first time my email client made an unsolicited suggestion about how to compose a thank you note to my grandmother. After seeing what it had autocompleted I thought, "oh, wow that's a lot better than I could do" for a split second before realizing, "wait, what the hell is wrong with me getting a computer to write a thank you note to my grandmother!"
I've never even tried to use AI to write since. I'd be so embarrassed.
Interesting question for sure, but for now the writing is just bad. So people might not know it's LLM prose, they just notice its poorly written and engage with the ideas less
Well, sure, but with his fly open the LLM user harvested more wheat. I guess everyone would walk around with fly open if it allowed them to do twice the amount of work or more.
I remember when I saw it the "list with things prefixed with emojis" the first time, and started doing it myself thinking it was a great way to break down things visually. Looking back at the things I wrote then people would definitely assume it's LLM.
I sometimes see posts on LinkedIn that look LLM generated but also makes me think it's someone who has seen a post, thinking it's a great way to convey a point and tries to adapt the style, without realizing it's a smell.
Whenever I see a cringe post on LinkedIn, I medically block that person or mark that post as not interested. That’s the only way you can judge the algorithm not to show those posts
We hired a new guy at work that's leading a specific platform initiative.
In person he's very articulate, able to communicate abstract thoughts clearly, states clear goals and how he's learning about the business to develop a plan.
But dear lord everything he writes in email and slack reads like it's copy paste from chatgpt. It's unnerving.
Had a similar experience when my talented lead engineer, who usually grunts in monosyllables and writes terse responses to my apparently intellectually frivolous questions, started adding flowery emojis to his email.
I'm gonna be contrarian here and say this is a Good Thing. It's very easy to tag to tag this sort of writing, and that makes it an absolutely great signal about hte people who post it. The more vacuous or frequent it is, they less weight you can accord them and the easier it probably is to get past their filters and make use of them for your own ends.
Is that morally bad? Guess that depends on exactly how you make use of them and what those ends are. If you an encourage an idiot to praise your competitor's product in order to get a reverse halo effect, that's kinda bad. If you give them some empty flattery in order to bridge a contact with someone you actually want to connect with, that's probably OK. The point is that uncritical LLM repetition tells you something about the person and lets you see past metrics like the apparent amount of wealth they have, the intimidatingly deep resume, or the degree size or centrality of their network. Two things, in fact: they post any old thing that brings in the clicks, and they're cheap. Previously many of these people probably paid someone to ghost-write their commercial affirmations.
Conclusory zinger goes here - punch up the dramatic contrast
PS I'm reducing your fee to 10c/word, hope that's OK. Inflation
I'm not morally/philosophically opposed to someone generating text with AI. I just don't want to read something that isn't really "owned" by the person creating it (that is, the writer knows what he is sending me and endorses all of its contents), and because I can't discern that from people I don't know, I tend to view it with suspicion by default.
The only place where I use AI for writing is to summarize press articles; this allows me to legally have a mirror of a press article I am commenting about in my blog (whenever I use AI, I make a disclaimer that the prose is AI-generated; EDIT: I also link to the original article, and an archived copy if available). Now that archive.org doesn’t mirror many press articles, and now that archive.today is having a lot of reliability problems (right now its CAPTCHA is down because it has gone beyond Google’s limit for CAPTCHA use), an AI summary of a news article (along with a link to the real article) is the only way to guarantee the reader will get this gist of the article I’m commenting on.
I also use AI to colorize old black and white pictures when discussing historical events in my blog, as well as the occasional AI enhancement of an old grainy and/or blurry photo.
Actually, it’s because, as per Feist Publications, Inc. v. Rural Telephone Service Co., facts cannot be copyrighted.
This legal trick only works for rewriting an article reporting on a factual event—since the events are uncopyrightable facts, the only part of the article which can be copyrighted is the stylistic writing.
Let me quote from a recent legal opinion on AI summaries (The New York Times Company v. Microsoft Corporation et al 2025):
>>>
Exhibit 11 to the CIR complaint provides website links to articles that
CIR alleges were unlawfully abridged by defendants in their ChatGPT and
Copilot outputs. (CIR, FAC Ex. 11.) Examining the similarities between
those outputs and the corresponding CIR articles, including the “total
concept and feel, theme . . . sequence, pace, and setting,” Williams
v. Crichton, 84 F.3d 581, 588 (2d Cir. 1996), the Court concludes that
the “abridgments” contained in Exhibit 11 are not substantially
similar to CIR’s copyrighted works as a matter of law.
The alleged abridgments are detailed summaries, usually in bullet point
form, of the facts contained in CIR’s articles. Those summaries—which
differ in style, tone, length, and sentence structure from CIR’s
articles—are not “substantially similar” to CIR’s copyrighted
works. They present the “facts in a different arrangement”—bullet
point lists or short summary paragraphs—“with a different sentence
structure and different phrasing.” Nihon, 166 F.3d at 71. In short, the
abridgments in Exhibit 11 are not substantially similar, qualitatively
or quantitatively, to the original CIR articles as a matter of law. The
Court therefore grants OpenAI’s motion to dismiss CIR’s claim of
direct infringement under 17 U.S.C. § 501 insofar as it relates to the
“abridgments” contained in Exhibit 11.<<<
Copyright only covers distribution, but I distribute the AI summaries on my blog. With paywalled articles and the usual archives having serious reliability issues, it allows readers of my blog to get context about articles I comment on.
The links are AI summaries, which, in turn, link to the original articles, but, in some cases, the original articles are paywalled. For the ones which aren’t paywalled, having a local summary prevents link rot.
Fair use covers quoting someone to comment on them. For example, in New Era Publications International, ApS v. Henry Holt and Co., it was ruled that quoting Hubbard saying “The trouble with China is, there are too many Chinks here.” was fair use, since the book in question was commenting on Hubbard’s personality, and could only reasonably do so by directly quoting him.
From that decision:
>>>
these brief quotations from unpublished copyrighted work display a compelling fair use purpose. [...] These quotations are in mockery, to show Hubbard's bigotry, bias and coarse lack of taste. This is not an instance of the biographer/critic free riding on the creative talent of the subject.
<<<
It’s never been easier to differentiate yourself as a writer. You just write with some personality. People don’t want writing that doesn’t look average, but looks novel.
I despair much less for human writers than a did maybe a year ago.
I used to do that, then someone on HN basically said I had mental problems because of the way I wrote the article. Best part is Claude told me to tone it down a bit, and I completely ignored its advice. Wouldn't have happened if I had listened to the AI.
I've also been called a schizophrenic on a GNU mailing list because of my idea and the working code I submitted. Caused me to literally quit the list on the spot. Best part is the maintainer eventually implemented his own version of it.
Since then, others have encouraged me to keep it up, but I just don't feel comfortable anymore with this "just be yourself" nonsense.
I think that's the first time someone claims that "write with some personality" is some easy thing you just learn somehow. There are authors out there, even ones that make a living on their writing, who still haven't learned to "write with some personality".
What exactly does that mean and how concretely can people actually do this in practice? A few "tips and tricks" might be more helpful than "just write better" or similar stuff.
Thanks a bunch! I actually tried "writing more personal" just some weeks ago, and the feedback I received so far been relatively OK I think, except some people apparently think the part about the drugs to be childish. May I ask what you think of it, if it does sound like it's written with actual personality? https://emsh.cat/en/the-people-are-still-there/
I won't claim to be a professional author or even good, but lately I've been trying to get more into the "it's a person who writes actually" direction and this was an exploration into that, so any sort of feedback would be most welcome, if you have the time!
Even worse in the corporate world. Most emails about some launch announcements are now entirely written by AI. TTFED (Time to first em dash) is usually like 30 words or less.
What once has been a thoughtful email trying to describe in few words why something is launched and how it might help you etc. is now almost a novel with more paragraphs than substance within the tool being launched.
This makes it almost impossible to stand out as well. Where in the past someone could create a grea looking announcement (eye-candy) and think deeply about what to write there, and then hopefully stand out in the sea of mediocre ones, now every little email seems like it's a multi-million $ SaaS being launched.
Just last week we launched an internal tool which was in development for months, and literally a handful of people even bothered clicking the links within the announcement.
This is being one-upped still by leaders writing big project plans for 4-5 months ahead, using AI. Everything from the inception of the project(s) is AI. It has bizzare timelines, more codenames than actual people working on it, the vaguest descriptions of what the things will do etc.
Then this is trickled down into the teams, and they... to no ones surprise, throw more LLM at it. Now they start working on the LLM project plan using claude etc. The end-effect is baffling in all sorts of ways (quality, ui/ux, all pages looking different), AI generated images and more.
And then, finally, they colaborate on big announcement emails using AI.
And if you don't share the optimism and try to explain why this is silly, you're an AI sceptic...
True story from within one of the biggest companies in the world.
The people who are going to be successful in the coming economy is the ones who can channel all the knowledge and experience of an llm without shame.
If you are smart:
I've been doing that. It helps that I have a substantial human writing corpus to draw from. Results are still mixed. Models simply can't resist adding some LLMisms. Pangram still detects some of it but not others, and also misdetects some of my actual writing as AI generated.
Honestly, I'd rather the shame and hate just went away instead. It's seriously exhausting and I'm starting to feel tempted to just give up and either start using AI more heavily or stop writing altogether.
Another AI-as-amplifier vs AI-as-substitute article.
Wondering what happens when tomorrow AI accepts and learns from the feedback and gets trained on all the failed initiative as well. It wouldn't be so hard if failed (something which wasn't right at all or hasn't got the traction) projects and ideas are all listed somewhere for an LLM to scan through. LLM may finally figure out how to add personal scars, hard decisions and personal insights which can be personalized further.
Also I don't think it might be so undesirable for an org, if there is a system which observes and present hard facts based on last quarter or year JIRA (pi planning and sprint planning) and commit histories.
It’s the best guy around who tells you when your fly is down, thanks.
I did writing courses as elective in college, the biggest lost truth is that there is no one correct writing style, (you build) being a writer at any capacity means cultivating your own writing voice, (yours) which is an expression of who you are. (voice)
Your fly is also open when you indiscriminately fling accusations of LLM writing around. I know it isn't that easy to detect because I get falsely accused of it. To some people it seems to mean text written in complete sentences without glaring errors and maybe a few uncommon word choices. It has become an all purpose excuse for declining to engage with the meaning.
One of my neighbors sent a letter to the HOA president, who refused to even acknowledge it, on the grounds that it was written too well and so must have been AI assisted. I don't know if that was true, and don't much care, because I do know it contained valid issues that deserved a response.
> Your fly is also open when you indiscriminately fling accusations of LLM writing around
Not.. really?
I kinda see the point you seem to want to make, but the "no U" opener makes it hard to do that.
Beside that, HOAs - from what I heard of them - will use any reason they can make up to ignore what you want from them, so I'm not sure if that has anything to do with LLMs.
I struggle with this. I think people just will not want to read what LLMs output, plus perhaps you can still analyze things better than an LLM, although that seems skeptical.
My situation is writing about futuristic long-term business strategy ideas; can that be replaced?
I have found LLM’s to be a really good thinking partner for writing. I start with a rough idea then proceed to argue for hours challenging ourselves and throwing away a lot of drafts until something materially useful emerges. It is a very different approach to writing… much morel like attaching a debugger to my thought process.
Also in the author's latest post[0], they took this[1] self-reported preference poll seriously, which makes it very hard for me to take their articles seriously.
No, you explained why the selection bias doesn't matter, which I agree. The self-report data still is completely worthless in the first place though.
People are really, really good at lying to themselves, let alone to an online poll. They'll tell you that they prefer imperfect or even bad writing as long as it's not AI slop, just like how they'll tell you they like healthier food, they prioritize personality instead of look for potential dates, how they use LLM "only as a spellchecker", and how they use tiktok for educational videos. As long as there is no stake, people will just say what make they feel better.
Self-reporting data for human behavior is just noise.
I mean, fine, it's anecdotal -- but the numbers are also overwhelming (and don't seem to disagree with the tenor here).
And look: you're obviously free to ignore me because you feel that the survey data is "completely worthless" and just slop your way to success -- all I'm doing is trying to explain why you shouldn't expect me (and people like me) to read what you create.
I don't agree. Good writing and good ideas are entirely different things. Just read all the beautifully written, inaccurate stuff by the NYT or the Guardian.
Also, non-native English speakers have to use LLMs to share their views so that they are not judged on their writing.
This post is also a good example; it's well written, and I enjoyed it. The idea could have been expressed simply as: "People can smell LLMs in your writing, it makes you look disingenous"
> I don't agree. Good writing and good ideas are entirely different things. Just read all the beautifully written, inaccurate stuff by the NYT or the Guardian.
I'd be very curious of some examples when great ideas are being presented in bad writing (or bad speaking for that matter).
> Also, non-native English speakers have to use LLMs to share their views so that they are not judged on their writing.
weird statement. If they use LLMs they will be then judged for both their inability to write in English and for their usage of AI. Not good.
One example that comes to mind is math textbooks – the early textbooks in a field are usually much worse than later ones that come along. To pick on one, I think most people who've read both would agree that the commutative algebra section of Lang's Algebra is much worse than Atiyah & McDonald's Commutative Algebra book, despite covering basically the same ideas, theorems, etc.
I think this is due to the fact that the language game that we have collectively decided to play, with AI, is not yet mature enough. In the language game of “giving a gift”, for instance, there is a distinction between giving a gift that you made yourself, and one that you bought at the store, but it is not that great, nobody cares a lot anymore. Right now, the language game of publishing words on the internet is strongly associated with the notion that these are “my words”, and the fact that they might have been written by a machine instead, introduces some discomfort. But it’s possible to imagine a future where this language game will have evolved enough that the notion of authorship by a particular machine entity, will not in itself be such an important issue, anymore.
Personally I never use an LLM or "AI" for any reason. Don't find them useful and, like others, don't find their writing compelling or interesting. I can understand using it as a coding tool, but can't fathom anyone wanting or needing to filter their own thoughts and ideas through it. Is it really that difficult for anyone to send a letter or an email without machine assistance?! If someone sent me a machine generated email I would view it as being no different than any other spam message. In my opinion doing so shows a lack of respect for the recipient whose time is being wasted with machine generated trash.
If you never used any particular tool, you probably wouldn't find it useful, right?
I asked Gemini about my unicycling this morning, and it coached me on weighting the seat; e.g., looking ahead instead of down helps unweight the pedals.
That's true. I've also never used a wheelchair, or a variety of other tools for people who can't walk, because I can walk without them. The same holds true for communication.
Much like our voices, DNA, odor, and personalities are unique, so is our writing style. When I read a wholly-human writing, I get a sense of what kind of a human I'm dealing with. Using LLMs will distort this signal, so readers worth their salt will easily skip the piece, because we don't want to acquire LLM-like writing 'style' (reading affects writing).
As soon as I sense Slop, I'm done, this person refused to think when writing, why should I waste my time reading it then?
I wanted to share my project, but for some reason, the site won't let me... I guess my karma is too low—ha ha ha! We're all like kids trying to get a good grade.
I think one of my first posts ever was a share. My guess is there probably a new account timeout to prevent spam, and you are affected for having made your account today
So many people seem completely blind to LLM writing, they are fine sharing it on social media or at work. I think it’s people who are “not details people” and are used to judging something without reading it based on superficial cues which is what LLMs are optimized for.
And of course anyone who actually reads stuff is disgusted by it, so we quickly end up with a situation where content is piling up and nobody is consuming it which is basically dead internet theory.
On LinkedIn specifically I went from reading it regularly to almost never touching it, previously the nonsense (“the interviewer was the dog”) was still tolerable enough to flip through for updates and I found the platform useful for business leads. Now it’s just a feed of pure slop, when I do open it I just close again after reading a post is two when I remember how bad it is.
Yes, yes, all is true here. But you forget: Half of America is illiterate. "According to data analyzed via the National Literacy Institute, roughly 54% of American adults read below a sixth-grade reading level."
So, those folks aren't checking anything, and certainly wouldn't know the difference, or care.
LLMs are not defacto bad at writing and are not defacto distinguishable from human writers. Lots of people are just lazy, never customize anything, and tell chatgpt to make them a post about x to be a "thought leader". Most are even lazier and blatantly rip off some else for their own means.
However, it is perfectly possible to have an LLM imitate an existing corpus of writing (yours or someone else) and with a good prompt, idea, and editing, to produce high quality writing (in every sense of the word) with an LLM.
If you're going to put that level of effort in, just write the thing yourself. It will be a comparable level of effort and you'll learn something doing it. Your readers will also appreciate hearing your real voice.
It's been odd as I've seen colleagues I generally thought of as intelligent and self aware post glaringly AI generated cliche LinkedIn corporate slop, even accounts from generally respectable and innovative tech companies. I don't know whether they think no one can tell or that they don't care if it is obviously AI? It's truly bizarre and now seems to be the vast majority of posts there.
Interesting quick read/take. Especially the LinkedIn analogy. Even though I am WAY high on the AI fanboy and low on the LinkedIn side of the bell curve, I can see what the author mean.
Wait, you're complaining about AI-generated content on LinkedIn? The place that was 99.9% organic human content slop even before there were LLMs? It was always all form, no substance.
People, such as the author, who are smart and competent--be it writing, math ,coding etc.--I think systematically underestimate the difficulty of said task for a general population. Things they take for granted, such as writing well, are surprisingly difficult for large proportion of the general population
Just like LLM-generated blurbs are now a cliched trope, so are short blog posts about people experience in a world with a lot of LLM usage. And this is one of those.
LinkedIn is also suppressive. The algo for maximizing non-confrontational screen time is strongly tuned against: critical posts and comments. Real bringing of information from outside the bubble. Links are severely suppressed.
To me, LLMs struggle with the most important aspect of technical writing - conciseness. As Einstein put it: “If you can't explain it simply, you don't understand it well enough”
I wouldn't ever use an LLM for my own writing. But I'd use, after edits, for letters to bureaucrats, landlords and other types I'm forced to deal with.
The thing is, the author may care about Linkedin but the relationship that many people have with linkedin is as a place they have to be. "Hmm, I need a blog to enhance my career but writing, urg. I know just the thing...". Which is to say, it's not strange Linkedin is going to be filled with crap. None of my actual friends on Facebook post crap 'cause there's no incentive.
The HN title ("Intellectual Fly Is Open") made me think it's about some startup starting to sell subscriptions. That title should be de-editorialized. Was it LLM-generated?
This feels like satire / parody. Praising LinkedIn. The text itself containing multiple LLM tells, especially in the parts complaining most about LLM writing.
Maybe not. Could just be kind of accidentally ironic with the author picking up LLM quirks from using them a lot.
I guarantee you people dislike my writing way, way more than they would if it was AI generated. People need to look in the mirror before telling others not to utilize LLM's for writing.
it's all about money, they write some attractive stuff, they get some views, likes follows, maybe someone will say "this dude is smart". but most times you get idiots liking the stuff, and likely detrimental because a smart person would see is slop, and they also know you don't know people know is slop.
I was fired from my job at spotify for taking fmla when i my dad was dying from cancer. I've been marking the my directors posts on linkedin as 'looks like ai slop'.
LLMs write better than most journalists. It also writes better than majority people in the world when it comes to English as majority are really bad in writing.
And of course, LLMs write as you prompt.
Personally, I'm happy to read broken English from non English people or good English from English people. But I'd rather read LLM writing than read typical long form perfect English journalist crap.
I used to think this, but now that I've been trying to use them to help with writing, it turns out they are not necessarily that great on the deeper level. The grammar and the surface style are impeccable, but they often struggle with continuity and carrying on a point or an argument.
I've come to trust their grammar corrections implicitly. So much that I've setup a "Fix Grammar" command with Obsidian Copilot that I'll just let it run on a whole chapter (YOLO mode):
Proofread the following text for grammar, punctuation, and typos while
strictly preserving the author's voice, pacing, and intentional stylistic
choices (such as sentence fragments or character dialogue quirks).
Rules:
- Correct objective spelling errors, misused words, and unintended punctuation mistakes.
- Smooth out unintentional syntactic snags without homogenizing unique phrasing.
- Retain all Markdown formatting.
- Return ONLY the corrected text with no intro, summary, or explanations.
Text to edit:
{selection}
About a year ago, all three big AI vendors could end up with strange "corrections" that would throw you off, but not anymore. As far as I'm concerned, they've solved "the grammar checking problem."
The complexity in this command is there for phi4:14b and qwen3:14b which I run locally via ollama. If the text doesn't have any Obsidian callouts or similar, fancy stuff I just use either of those and they do a fantastic job in seconds. For Big AI (e.g. Gemini, GPT-whatevs, Claude) you can literally just tell it, "fix the grammar." No need for the lengthy command.
NOTE: I am decent with English grammar so most of what needs fixing is typos I didn't spot or misplaced commas and periods inside/outside of quotes (I always screw that up without thinking—even though I know the rules! LOL). Occasionally, Big AI (Gemini, specifically) have disagreements about whether a comma is necessary in a particular spot but it's always of no real consequence.
> About a year ago, all three big AI vendors could end up with strange "corrections" that would throw you off, but not anymore.
Without a history of diffs created by this method, I don't believe you.
> I am decent with English grammar so most of what needs fixing is typos I didn't spot or misplaced commas and periods inside/outside of quotes (I always screw that up without thinking—even though I know the rules! LOL).
The rules there vary by style guide and are not objective.
LLMs seem like massive overkill for something the red and green squiggles can already accomplish.
Good writers break grammar rules when it helps convey a point, emphasize an idea, or just makes the writing more interesting. You are completely missing out on that.
> But I'd rather read LLM writing than read typical long form perfect English journalist crap.
The problem is precisely that LLM writing is that exact thing (at least by default), but even more so. Longer-form, more "perfect" in some technical sense, and yet crappier.
I suspect the commenter prefers the concise, factual summary of an LLM.
It is true that modern journalism on the web is trying to get ad impressions. After all, this is how we ended up with clickbait. Sure you still have the Economist and Atlantic (which aren't beyond criticism), but your local county paper is an absolute mess. The content is stretched, meandering, and designed to keep you scrolling through more impressions.
I would almost never transmit LLM output without a decent amount of editing. But I also don't understand people who paint it as this horrible offensive unreadable slop.
Calling it better than most journalists is extreme (and probably the reason for down votes), but it must certainly be better than the average person.
We're hearing this criticism from the highly educated professional class people who bother to have their own blog or otherwise spend their time talking about technology online. I mean come on.
To most people, the LLM must feel incredibly empowering, like us wearing a mecha suit. Would we always show restraint and only apply our newfound enhanced physical strength in carefully considered situations?
> But I also don't understand people who paint it as this horrible offensive unreadable slop.
When a piece of writing is full of LLM tells, I find it as offensive as any formulaic writing, except that the LLM style has quickly become pervasive, much more so than any other type of formulaic writing.
I’m not reacting out of some general dislike of LLMs - I use them daily. I’m reacting because I don’t like terrible writing.
Well said. When I realize I've been reading LLM filler and leave the page it's the same reaction I would have pre-2021 thinking I found a good source for something and realizing I'm actually reading blogspam.
The difference being this variant of blogspam is produced with basically zero cost and thus the infection has spread far beyond SEO into every UI surface with textbox. Plus there are now passionate defenders who insist finding their blogspam unpleasant to read is disrespectful or anti-progress somehow.
The message of the post is what _should_ hopefully matter to a reader.
And, not the fact that it was written by an LLM.
I am a technical guy at heart; also an introverted extrovert. I hate writing docs that are to be written to satisfy someone else's metrics. For it to be a tickbox'ed item.
I delegate that to an LLM. I want to spend my time solving interesting challenges instead.
So, Mr. Cantrill, you got a problem with that? So be it.
> Please read past the LLM-ness of any given post.
I just can't. LLM assisted posts are often so long, and the hints are usually obvious right at the start.
They're so unpleasant to read. "It's not X, it's Y" etc are bad because the comparisons rarely add anything at all to the message. It's filler that wastes my time. At that point I'd rather see your original prompt.
Why should I invest several minutes of my day reading something if I've already seen evidence that the author doesn't respect my time?
It's not that an LLM wrote it so much as the author couldn't be bothered to clean it up and remove the garbage cliches before publishing it.
I hear ya. However, that ship has sailed a long time ago and succumbing to this very idea is exactly why puritans in any form will not survive the next 50 years.
> Please read past the LLM-ness of any given post. The message of the post is what _should_ hopefully matter to a reader. And, not the fact that it was written by an LLM.
No. I am interested in what a human has to say, not a clanker. If you don't wish to write that is fine, but don't hand it off to the slop machine and present it as though you did anything of value.
These are all important points and I love the analogy. But there is an even bigger issue with having LLMs write for you:
Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
Reviewing gives you the chance to ensure the arguments connect solidly, that references are accurate (even informal references) and gives you the time to consider counter-arguments you aren't addressing.
None of this matters much on LinkedIn, but it matters a lot in our work. You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
> Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.
I don't disagree, but I think it's often not appreciated how much there's other work to writing too.
The biggest one is that you have to communicate non-interactively to an unknown audience. Having to (literally) put it in someone else's assumed terms does help giving different perspectives into the matter, but doesn't necessarily help one's own thinking that much. Instead you have to do some of the reader's thinking for them.
You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Not all writing is thinking and not all thinking is writing.
Strange! I would quite specifically highlight “having to do the reader’s thinking for them”, as well as more generally developing the skill to “communicate non-interactively with an unknown audience”, as extremely valuable upgrades to my thinking.
It's a kind of shared fiction of an imaginary person's thinking, really. The author writes "now, I know what you must be thinking", and hopes the reader will agree "OK, close enough".
> It's a kind of shared fiction of an imaginary person's thinking, really
That is the best way I’ve seen anyone put into words what I feel about those types of theses
Most valuable thinking is done at the margins, where you don’t have much capacity to emphasize with a diverse and unknown audience.
That said, abdicating to an LLM is the worst of all worlds - you’re not thinking and the product is not tailored.
The solution is obvious - write as much detail as you need and allow readers to interrogate the virtual you with an LLM, maybe not even reading what you write.
> You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.
Hard disagree. Constraint is the driver of creativity. Also rewording sentences to sound better or make sense can make you reconceptualize the whole concept you are expressing
Sure, but doing an interpretative dance or an abstract painting can make you reconceptualize the whole concept too. But we're not really pushing those tools as much as writing.
Take for example a non-native writer of the language. I'm sure having to check up words from a dictionary may help to reconceptualize things, but I'm sure also that it's not often very efficient. And I think similar is going on for natives too for many types of writing.
It’s not clear that doing an abstract painting can make you reconceptualize in the way that reframing in other words does. The point about putting in other words is that you may stumble on a clearer, more tractable, more extensible framing. The kind of reframing an abstract painting does is very different, more like changing your attitude or way of looking. But I don’t think it ever leads to a place where you will suddenly find yourself with a sharper understanding that helps you communicate with others more effectively.
I'm not much of an artist, but I'd guess trying to paint a painting of how quicksort works, in a way someone else can grasp it too, can need a lot more reconceptualization than writing a description about it. Doing plots or diagrams, or even implementing an algorithm, for sure often need more thinking than writing a description.
Yes but diagrams aren’t abstract paintings (or interpretive dances), which I assume the original commenter meant as intentionally artsy and indirect forms of art.
All writing is thinking when done by a human, you’re literally distilling your thoughts into words. You can’t write without thought.
And not all thinking is writing is a clear truism, there’s no point to even stating that.
Writing helps us think about the world, it’s a pivotal intellectual technology.
> Writing helps us think about the world, it’s a pivotal intellectual technology.
Much like money decoupled selling and buying to move away from bartering, writing decoupled saying and hearing so they didn't have to happen at the same time. The incredible step that happened was not that people had to think a whole lot, it was that thinking that was already happening had to happen once.
> All writing is thinking when done by a human, you’re literally distilling your thoughts into words. You can’t write without thought.
Of course you can. You can write down exactly what you hear, for dictation.
You can write down a stream of consciousness and put barely any thought into it at all.
I can't help but feel most here are massively over estimating human writing. Human writing is, almost universally, terrible. We have entire jobs that are hard to fill just to make things sort of ok. Good writing is a small subset of human output.
This is an example of the problem. I did choose to write it in the way that although I knew it can be interpreted trivially, if the reader assumes I'm an imbecile. It was a "punchy" recap of the relatively long explanation for a HN comment and hopefully decently argued point.
I have been thinking over this comment for a while now. I think yes, you are correct that thought is involved in writing but I don't think it is possible to claim "all writing is thinking when done by a human". Like what type of thinking are you claiming here?
Because one can copy a text and write it down and that involves thinking in the sense that anything we do involves thinking fundamentally. But that thinking is different from thinking logically about a concept and writing it down which I think is where you are getting at.
The definition of writing and thinking is too broad in that sentence even though it does apply in several obvious cateogires within that at different levels.
And also "writing helps us think about the world" is too broad again. Why? Why does me writing "apt apt apt apt apt apt apt" help me think about the world? I just wrote it because i felt like writing it. Why wouldn't you consider that writing?
> Why does me writing "apt apt apt apt apt apt apt" help me think about the world?
Poor example. You wrote it to make a point, after all.
Yes, but not to understand or think about the world. In this case, writing was to express something, not to understand something. The overall point is that claiming that "human writing = human thinking" is too broad a general statement to make seriously.
> The overall point is that claiming that "human writing = human thinking" is too broad a general statement to make seriously.
Do you have any non-contrived example of writing that was done with zero thought?
You wrote "apt apt apt apt apt apt" to prove a very specific point no? Absolutely requires reasoning and understanding the problem to go there. And good luck getting an LLM to do that.
> And good luck getting an LLM to do that.
Have you never asked a decent model to explain something to you? You should try it.
Yes, I wrote it to prove that writing doesn't mean I understand the world. It responds to "writing helps you understand the world." My point was that, yes, of course you can find categories within it that apply, which is why the answer is not fundamentally incorrect. Claiming "writing = thinking" in a general sense is very broad, and that's my point.
> Not all writing is thinking
Would love an example where you’re able to write without transferring your thoughts. Besides the obvious: fjcjfjrnjfjfifjfnrnakosifnrbwkofgjrj
I can think of the following:
(1) Translation from one language into another
(2) Transcription from one medium (audio) into another (text)
(3) Deception to obfuscate your thoughts
(4) Posting things like "First!", "This.", "Just google it.", etc.
(5) Textbook answers with no original thought.
> Translation from one language into another
Bad example, because translation is deeply creative. You can't blindly mill one language into another, because words and phrases and concepts and cultural references in one language frequently don't map 1:1. You have to find a way to convey meaning as closely as you can and not necessarily the words.
I think you meant that translation is, case in point, based on inference. The person doing the translating is inferring, based on context, intent and meaning and imparting that on what their output is. That's generally not desired from a taxonomy perspective.
That's called localizing and I can't stand it.
I prefer when things are kept 1:1 as is and maybe there's an explanation for things that don't quite make sense as a footnote.
Amusingly I absolutely abhor translations done by people who think the way you do. I want a translation that's as literal as possible and which provides the necessary commentary for me to understand any alien concepts, idioms, customs, etc. I absolutely never want "translated" cultural references. At that point the "translator" is nothing more than a shitty fan fiction author as far as I'm concerned.
Of course the above requires actual work on the part of the consumer. I realize many don't want that, particularly when it comes to entertainment. So I appreciate that the other sort of "translation" exists but I think it's important to realize what exactly those are.
Thankfully LLMs are more or less to the point of providing what I'm after in near real time.
But you aren't disagreeing with your parent comment. You seem to have a strong opinion that you are; that opinion is incorrect.
Words refer to a broad semantic region, a phenomenon technically known as "polysemy".
The range of a word in one language is always different from the range of analogous words in another language. This is a classification problem. And a translator must think about how to solve it. Imagine a Venn diagram with 20 circles that each overlap the other 19 to differing degrees. What does it mean to designate one of those circles as "the literal translation" of a foreign word?
Of course nothing involving natural language and human culture is exact. A "literal" translation is obviously a slightly fuzzy concept that speaks to intent. However I think I provided enough context that this should be clear. I gave the example that I don't want "translated" cultural references or idioms but rather the (approximately) literal wording and some commentary from the translator providing the necessary context.
Obviously there are degrees to this and obviously preferences will vary. I acknowledged that.
My point is that nothing in your comment was in any tension with the comment you responded to, and you still don't seem to realize that.
I think you've misunderstood something. The comment I replied to specifies "words and phrases and concepts and cultural references". You only seem to be talking about words (and perhaps exceedingly simple turns of phrase). I was quite direct that I disagree when it comes to (among other things) the more complex idioms and certainly when it comes to any and all cultural references.
Imagine a localization attempting to replace a reference to an actor, political scandal, or other concrete cultural reference from one country with the "equivalent" from another. I've encountered that sort of thing before and while there are certainly those who appreciate it I am emphatically not one of them. As far as I'm concerned that's shitty fan fiction.
There are also a lot of examples in most (all?) languages that rely on repetitive sounds, easily mistaken words, or other strictly auditory features of the native language. You literally cannot translate those things. I do not want shitty fan fiction, I want an explanatory note.
Here's what I understand:
- You've got a pet peeve.
- You're going to rant about it, because you want to, whether or not it's relevant to an existing conversation.
- You didn't bother to think about my comments.
- You didn't bother to think about habinero's comment either.
Here is the same passage of the Analects (part of the chapter Gongye Chang) in different translations:
--- Annping Chin ---
Zilu said, "We would like to hear what you would like to see yourself accomplish."
The Master said, "To give comfort to the old, to have the trust of my friends, and to have the young seeking to be near me."
--- David Hinton ---
Adept Lu then said: "No Master, we'd like to hear your greatest ambition."
"To comfort the old, to trust my friends, and to cherish the young."
---
Our focus here is on the second line, what Confucius says. Does he want to trust his friends, or does he want his friends to trust him?
Does he want to cherish the young, or does he want them to cherish him?
We might also ask, though the translators have agreed on this point, whether he wants to comfort the elderly or for the elderly to comfort him. (And we could further ask whether Confucius wants to personally comfort the elderly, or whether what he has in mind is for society in general to do that.)
All three clauses are formed the same way in the original Classical Chinese, and for a couple of interacting technical reasons they are all ambiguous in this way. Translators, as you can see, make different choices.
But of relevance here, when you're doing a translation to English, you have no option but to make a choice. It isn't possible to render the original text 'in literal translation' and append a note explaining what went wrong. You must commit to a meaning behind the text and phrase that meaning in English. You can also append a note explaining that you might have chosen wrong, but English simply doesn't allow you to do anything that parallels the source material.
I really think you aren't talking about the same thing that I am. I'm not sure why you're assuming bad faith on my part rather than engage in discussion to clarify.
I think it should be quite clear by now that I am not talking about isolated words that broadly lack an equivalent concept in the target language. I even quoted the bit from the original comment that I took issue with and proceeded to give examples so I'm really not sure where the misunderstanding between us could lie at this point. Perhaps you are the one who should stop and more carefully think about what I wrote?
As to your example. I certainly do not accept that this is a case where we should throw our hands up and accept that different translators will go about things differently. Those two sentences in english have (as you note) rather different meanings. So either one or both translators must be wrong.
You have indicated that the original work in the native language is ambiguous. In such a case I do not think it is remotely acceptable for a translator to arbitrarily pick one of several possible meanings and just run with it. If the original meaning of the text is ambiguous then removing that ambiguity changes the meaning thus it is a bad translation. The translator instead needs to faithfully communicate that ambiguity, possibly resorting to a note if it isn't possible to easily express such a thing in the target language.
I realize that many people aren't going to want such a marked up copy. But without all the gory detail the reader will be consuming some sort of bizarre partial fan fiction. Your example illustrates that perfectly.
Did you hear the one about Scots Wikipedia?
https://www.theguardian.com/uk-news/2020/aug/26/shock-an-aw-...
I did think about this while writing, and I made the compromise to accept that someone will nitpick about it to hopefully drive the point better for those willing to read it charitably.
QED?
What is the charitable interpretation? I am not being coy or sarcastic here; it is genuinely (forgive the claude-ism) unclear to me what your arguments are.
With the last line I tried to condense (and oversimplify) about these ideas:
Not all writing is thinking: That all, even a lot, of writing, or parts of writing, is such that it will develop one's thinking much. For example most stuff I have to write, the dozen emails a day, the funding application boilerplates, the reports are stuff that don't really need (or deserve) much thinking but they have to get written. And even in the writing that deserves attention, there is stuff like grammar and spelling and surface style that usually take quite a bit of time after the ideas have been written down already.
Not all thinking is writing: for many cases writing is not a particularly efficient way to develop one's thinking, and e.g. visualizations, math, coding, discussions etc can be a lot better.
When you transfer other people's thoughts. On this board, we are engaged in pursuits where the truth matters. A lot of writing is about showing that you belong to the right in-groups. That is better done by repeating their talking points than sitting down and coming up with an earnest way to show that you agree.
The number of times the friction of writing has saved me from prematurely communicating a poorly understood idea must be in the hundreds or maybe thousands. For me when something is difficult to write about, it's a very good signal that I don't understand it well enough. So I think all the things you label as aspects of writing that aren't necessarily "thinking" are nonetheless good for thinking because they provide some necessary friction.
This to me is the key reason people are so obsessed with using an LLM for everything.
They’re completely opposed to experiencing any type of friction.
Hence The Great Smoothening of Minds we're all experiencing in this decrepit era. I blame the financial incentive, and welcome its disappearance. There are too many people doing computers just for the big paycheck, it would be more fun without them. Maybe the AI bubble popping will get rid of them.
(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
It reminds me of the transition over the last year from AI-assisted coding to AI doing all the coding. At first the code output wasn't good enough, and humans read and iterated on the code all day, so the details of the source code mattered. Now, the code is largely high quality and it meets a large set of guardrails we've set up over the years (linters, typecheckers, security checks, LLM-assisted code quality checkers), and it's just Claude working on the code, so the details matter less and engineers think a level or two up (machine code < assembly/bytecode < source code < conversation with agent < artifact with high level design).
I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document. But writing and coding are different enough in a number of ways that this is far from inevitable.
> wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
What use is that? I'm not being facetious, I'd really rather like to know.
Who or what is the audience for that sort of long form writing? If it's a human, why would they read it? They'd just give it to an LLM and get the salient points back. If the audience is another LLM, why expand it?
The only use case is an audience of humans who still read and understand, and those people aren't going to be interested in a message when it is not apparent that the sender actually understands the message themselves.
A lot of corporate documentation exists solely to measure if people are working or not.
That's why there's so many meetings in white collar companies. Because people can't understand what is going on at those documents so they just need to "align".
LLMs are amazing at generating this useless documentation that goes absolutely nowhere.
> A lot of corporate documentation exists solely to measure if people are working or not.
That's already available today. We don't have to perfect LLM writing.
You can imagine:
1. I have a bunch of data or research that I've gathered with a unique hypothesis
2. Having gotten my arms around that pile of information, I believe I have a compelling thesis to put forth
3. I design the narrative arc and of the thesis. The important parts, the necessary but not sufficient scaffolding.
4. An AI helps fill in the story from there. Fact checks each claim, connects the dots, makes it comprehensible.
Who is this for? Well, quite possibly the human who asked for it. It's pretty informative to read back a research brief in full that you helped do the scaffolding.
Also of very clear use is other AI's who did not have the same unique hypothesis and did not gather the supporting evidence. It's an interesting angle for others to build on.
And of course, other humans! Most human written content gets almost zero readers today as it is. And I suppose LLM content probably pulls the asymptote closer to zero, but some pieces of content may be genuinely interesting or useful.
> An AI helps fill in the story from there. Fact checks each claim, connects the dots, makes it comprehensible.
I think this certainly has some value but this claim in and of itself is stated like your hand-wavy step 3. How do they fact check claims and connect the dots?
Maybe LLMs get there but currently they write in an extremely verbose manner, and things that have gotten into the context window that are no longer relevant continue to stick around (just try having it write some code, then work some of it back to simplify the problem - it will insist on writing comments about code that no longer exists).
Right now using an LLM to write documents is like taking a superhighway to travel 100 meters. Yeah you're doing a lot but is all that really necessary?
I won't deny that LLMs will never have a place in writing. But I personally don't think the current form is "the one that actually lands" (!).
Then the reader can use an LLM to compress it back, and you can then interrogate it for details.
I used to say this was the future of advertising (cr sales person prompts “we have some new EV SUVs on the lot”; GPT generates an ad email with a synthetic video, blinking text etc; then the recipient’s spam processor tells them “that dealer has some new SUVs”. I suppose the same could happen with so-called “long form”.
> What use is that? I'm not being facetious, I'd really rather like to know.
People are terrible at writing. Near universally bad. Even good writers have drafts and editors.
There is a constant refrain here that somehow short messages are more valuable than longer ones. But that assumes it's understandable. Lots of short content is, frankly, awful because the writer cannot put themselves in the position of the reader and explain all the things around the point they're making that the reader really should be told.
You can view writing as translation. From your language to a language your audience speaks. At that level is it so odd if the word count differs from one side to the other?
To turn bad short writing into good long writing you must necessarily change the content of the writing, not just the shape.
Not necessarily. Poorly worded, ambiguous, confusingly ordered writing can be massively improved without changing the core content. Better setups and explanations can be longer without changing the message or meaning.
Look at it the other way, could you take a good longer message you’ve written and make it shorter and less readable for your audience while still making sense to you and containing the key points?
But this is the same with coding. The reason that AI can write code from a description that is shorter than the output is in large part because it makes decisions about the behavior that were unspecified in your prompt. We accept this for coding apparently, I guess because those decisions are often unimportant. We might accept it for writing too. I hope not.
wasn't it thoreau who said "Not that the story need be long, but it will take a long while to make it short"
personally, I think there's a time and place for short versus long, just like there's a time and place for a 45mins TV episode versus a 2 hour marathon movie.
You claim writing is thinking, but imply writing code isn't thinking.
My opinion of LLM design review isn't that high - it seems to miss design tweaks that could vastly simplify corner cases. But if your code isn't written for human consumption maybe it doesn't matter. I'm still directly responsible for what I commit, so I can't just offload it to Claude.
That's something I struggle with, I try to get LLMs to output code I don't care much about and focus on the parts I do and it kind of works but the problem is that reading code written by the LLM is even worth than reading LLM generating text. It's nauseating and you still have to read what the LLM did if you want to really work on the parts that matter.
I still write code, I just write it in English now. It's still rock and roll to me.
That... isn't writing code.
Real programmers use a magnetized needle and a steady hand!
Ops analogy is like their boss claiming they write code because they told you what to work on and you wrote the code to solve your bosses problem.
Relevant comic expressing the same idea: https://www.facebook.com/twonks/posts/great-job-/13130659843...
People have been "writing code" like this for decades. That a programmer happens to do it doesn't turn it into writing code, no one would have made that claim 25 years ago, and people who aren't programmers wouldn't make that claim today.
Sigh.
It's like watching somebody about to be hit by a bus. You yell, you wave your arms, but they either don't hear you, or they don't believe you. The last thing that goes through their head is a Greyhound's hood ornament.
Yes, it is. But the important question is, which person is the one standing in front of the bus?
That would be bigstrat2003. He's not a big fan of updating his priors.
You seem to be not picking up on the subtext here. Others are, though.
There is no subtext to bigstrat2003's argument, other than "I'm wrong, and I don't care." If you give persistent, repeatable instructions to a computer, you are programming it. If you disagree, you are gatekeeping. It's that simple.
The most popular programming languages in 2030 will, in fact, be English and Mandarin. Deal with it and get over it.
----------------
Edit, to bcrosby95: Look up the etymology of the word 'computer'. It didn't originally have anything to do with hardware. The first computers were people, who were told what to do ("programmed") without necessarily knowing what they were working on in a big-picture sense.
There is subtext to my argument, namely that you're the one standing in front of the bus, and trying to pull the rest of us in front of the bus, while shouting at the people standing safely out of the road. I'll continue to reject the replacement of humanity. (That doesn't mean remaining ignorant of AI as a technology. It means rejecting the idea that we should be thrilled to be replaced by AI.)
LLMs do not produce repeatable results, by their nature. Two people can give the exact same prompt to the exact same LLM and get different results. It's not a straightforward 1 + 1 always equals 2 process. This is like telling someone else to code something for you. You relinquish control of all the details.
Calling it gatekeeping is just laughable. That's like saying it's gatekeeping to say that the painter painted their painting, and that the person who commissioned the painting did not paint it. It's wholely absurd.
Anybody can pick up a book and learn to actually code themselves. Or you can use an LLM to try to make things without bothering with that. But even if the LLM worked perfectly, pretending these are the same thing is silly.
The history of the word computer is obviously irrelevant. Words change, it turns out.
It isn’t really by their nature that they don’t give repeatable results, right? They’re just a bunch of math, but they perform better with randomness injected so we choose do to so.
(I’ve heard about some GPU compute nuance meaning that even without randomness injected they still wouldn’t quite be deterministic, but that’s also not core to their nature)
LLMs do not produce repeatable results, by their nature.
Neither do humans.
Two people can give the exact same prompt to the exact same LLM and get different results.
No one cares.
This is like telling someone else to code something for you.
Exactly.
Still getting technical interviews in pure python, specifying you don't use LLM, in late 2026. 4 years now from me having to learn mandarin I guess...
> The most popular programming languages in 2030 will, in fact, be English and Mandarin. Deal with it and get over it.
What are you willing to bet?
I'm game. Say $1000, donated to a charity of the winner's choice? How do you want to set the bet up, and how do you think it should be decided?
To be precise: I will bet that high-level programming languages won't be any less popular as a whole, but the vast majority of code will be written by AI rather than humans, working from specs written in natural language or something very close to it.
What we call "source code" today will be thought of as "object code" by 2030. Something that occasionally needs to be inspected by humans, but rarely authored directly. Anyone not writing code this way had better be doing it as a hobby, because almost no one will pay for it.
Seems like a case for https://longbets.org/ You and the other poster just need to find some agreeable benchmark so you can decide who won.
Good idea, I know they've been around for a while. I just signed up under the same username in case 27183 is interested.
These arguments are so silly.
Other person probably doesn’t like the idea that LLMs will replace hard earned skills. On the flip side, I bet you’ve seen your skills atrophy at an alarming rate and are trying to justify it.
Both sides come from fear. Just relax and take things as they come. Whatever happens happens.
Expands into a document for _who_ to read? another LLM to re-compress?
It's baffling you people are in control of such a strong product when you are obsessed with this intellectual pornography; wow - look at how smart it made my thoughts look (n.b. look, not read). Don't look too close. And certainly don't ask me what it means.
You are just talking about the output though. If you only think at a "higher level" you aren't doing the actual thinking. Its the same with code. The output may be good enough, but over time you lose touch with the details to the extent that you can no longer serve a useful steering function for the organization. Before coding agents I'd seen this with many humans when they get promoted passed the point where they work with code directly and can't figure out how to add value there.
Yes, of course. At some point the LLMs will also run the organization.
In the meantime, they aren't capable of doing it, and neither are humans who never understand the code.
So, you believe that Sam Altman understands the code?
No, but he also doesn't steer technical architecture & engineering, does he?
Given that OpenAI is largely vibecoded these days, do you think anyone there really understands the code?
Many programmers don't write long text; their way of getting a deep understanding of a problem domain is to build something, is to write code - in a process very similar to writing a long piece of text - it has the same reflection and externalization of thought.
The widespread introduction of LLM code generation is very destructive to that.
Perhaps LLMs can be brought to support human cognition in the same way writing can; but that has yet to be designed and it does not seem to be the way things are heading.
Miss me with this. I write way more now than a year ago. Mainly because I have to explain myself to the LLM. I'm fairly certain I write 50x more than before simply because before my weekend was spent gaming and watching anime. Now I'm having fun building things.
LLM prompts definitely don’t feel like “writing” to me in the way that writing a blog post (or even an HN comment) does or writing code used to. There is nothing to work out, you don’t really need to think. You’re _typing_ sure, but I don’t think it counts as writing maybe because the text that you write is thrown away.
I am doing memory research on emotion, sensory perception, and cognitive quality over time while explaining in extreme detail how to do a specific hinge animation in another thread doing blender animations for mechanical movement. I am also writing blog posts manually about all these things with zero LLM help. I do this on purpose knowing the quality of the LLM output is directly proportional to the quality of the input.
I'm way more articulated now than I've ever been, because in the past I didn't have to -- I never write a blog post -- but I've learned that well thought-out writing, with clear description, will produce better code.
The bottom line is, prompting is definitely writing.
But what are you writing? My bet is it is short bursts of text that are thrown away as soon as it is interpreted by your LLM.
Idk man I get a ton out of gaming and other media. If I say “I’m a cinephile” no one bats an eye, it’s seen as elevated and intellectually stimulating. Your examples reflect more on the gate keeping we do with what’s considered “worthy of our time” and “art” than the value of LLM’s.
I'm not drawing a moral equilancy here. You're right that people get all gate keepy. But the premise was that I do less writing. The truth could not possibly be further from reality. My actual words typed is thru the roof.
You and the AI agree, but how about you and the team who will eventually read and do code review.
Cognitive burden increases marginally with AI assisted coding.
This is why we haven't seen big projects(think browsers and browser engines) spawning in the past year.
Indeed. I'm curious what the security team has to say about that approach, for example.
If you get an AI to review the code especially for security, it does a very good job st finding issues. Better than any human reviewers I have worked with, and getting better. As someone who works in security , I feel much less worried about security bugs on code reviewed by a AI for security issues, be it written by AI or human.
extra reviews never hurt. But trusting only the AI to both code and review security-wise? not even close to usable
It's anthropic, of course the entire team is also using AI to do the code reviews.
> but how about you and the team who will eventually read and do code review.
Why are you reviewing AI code in detail? Do you also review the assembly output of GCC line by line?
Compilers like GCC are deterministic and the source code already fully defines the behavior. LLMs are non-deterministic and will accept ambiguity, filling in details where you haven’t. These sorts of comparisons aren’t really fair.
In the case of writing, it’s like hiring someone to write a book for you vs. hiring someone to translate a book you wrote into another language. In the first case, you didn’t really define the message for readers, whereas in the second case you did, and the translator is converting that same message for another audience to consume.
Sure, but I don't know what GCC's behavior is, and I don't vet behavior differences between compiler upgrades. As long as the output works, why does it matter that the black box is deterministic?
It doesn’t matter how exactly GCC works, as long as the behavior is deterministic and consistent (GCC is likely maintaining backwards compatibility between versions, so the behavior of your code likely hasn’t changed). In that case, you can reason about the behavior you need and write your code appropriately.
In the case of LLMs, the behavior is non-deterministic and inconsistent. If I don’t explain how handle an edge case or give a performance constraint, the LLM will still produce code and may do so in different ways, handling edge cases differently and with different performance characteristics. I can’t reason about how the LLM will fill in those gaps, it’s “random.”
Maybe you don’t care about how the LLM handles those edge cases or handles performance, but that’s different than a deterministic abstraction whose implementation details you don’t care about, but whose logic and performance is deterministic and consistent
Because _someone_ has vetted the output of GCC. It’s used in flight-critical stuff.
The closest thing we have to vetting LLMs is “whoa look, it escaped this sandbox, that’s prolly not great but it’s so cool!”
Sure, I manually test the output of the LLM. Manual testing is actually the main role for humans doing software engineering these days.
I wouldn't use it for flight control software yet, at least not without careful review, but most software isn't exactly critical. At the same time, I wouldn't trust flight control software that was only reviewed by humans, since AI is so much better at debugging.
We'll probably need humans in the loop for safety critical software for at least a year or two, before AI fully outpaces humans at generating correct code.
> I wouldn't trust flight control software that was only reviewed by humans
So I assume you don’t fly? Or is it only software created after 2025 which must be reviewed by the All Knowing Entity?
Can’t imagine a client allowing me to pass the buck like this.
How so? As long as it works to spec, I haven't had anyone care. They literally hire people so they don't need to care about the details. Put money in, get working software out.
And, AI is rapidly getting better than people at both code review and authorship, so a human deeply involved is turning into nothing but a slowdown. The main purpose people have is testing that the specs were, in fact, implemented properly.
I guarantee your specs/testing are either inadequate and/or you're not leveraging lots of existing (and probably free open source) code that was already written by humans and meets the spec better without ever needing an LLM.
The vast majority of properly written software was already plumbing well over a decade ago. The software engineering is making high level decisions based on experience with respect to the existing tools and the needs of the business. If you're not already using LLMs that way, you would have been a similarly bad manager of human devs writing similar inadequate slop. Less code has always been better code.
The line in the sand for these arguments really ought to be whether you think LLMs are better than humans who actually know what they're doing.
If you think LLMs are better, or could get better while continuing to use statistical methods, you automatically lose the argument (delusional/ignorant) and any hope of regaining credibility. That's not dogma. That's the science.
> That's not dogma. That's the science.
twitch
You're absolutely right to react that way.
For instructions you really care about, yes of course you review the assembly output! Usually when you're doing SIMD or want to check atomics are doing what you expect.
You can do that with LLMs for the parts you really care about too. The LLMs aren't regenerating the codebase from scratch every time, so the results stick around.
Only if it's not too byzantize to understand what part you should look at
The AI can tell you where to look. It's really good at this. It's actually a lot better at analyzing code than it is at writing it.
I can’t remember the last time GCC emitted code that just flat out called the wrong function. If it did that occasionally, I would review it.
Come on, this is a take we expect from a 1st grader!
Gcc makes maybe 1 mistake ever 2 billion emissions. LLMs make 1 mistake ever 3rd emission.
No, because GCC doesn't randomly fuck up the assembly generation (much less on a fairly frequent basis the way LLMs do). If it did, you bet I'd be reviewing the assembly line by line, or decline to use such a poorly performing tool (as I have with LLMs).
> You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
What a horrible, cold, inhumane world that would be.
I mean these are cold inhumane companies and their employees reflect it. SF is truly where human ingenuity goes to die, truly a blight on the industry as a whole and holding us back tremendously.
Writing is already amenable to many different levels of abstraction, though. If an LLM can expand your outline into writing, then you aren’t writing at the correct level of abstraction in my opinion; you should instead be explaining how you arrived at your chosen outline. You don’t need to explain the details because any party can generate those with an LLM; same as how many PRs today can be auto-generated and no one needs to read implementations; that is no longer the correct level abstraction to work at. This should actually free us to do work at a higher level of abstraction —- more consideration of strategy, objectives, etc and less worry about implementation details.
This only applies if LLMs aren't making mistakes 20% of the time and that's the problem. When you're only saving time on the easy part, it doesn't matter if you're working twice as fast because review of the tricky parts is still going to take 80% of what it would have taken to do the whole thing. Total effort ends up being more rather than less if you want the same quality.
> it's just Claude working on the code, so the details matter less
If you're not billed for usage, anyway.
Otherwise, for the other 99% of folks, that attitude is of course a pit trap that captures code bases and makes them maintainable only through the providers -- presumably one or few -- with a rich enough model to keep up with the growing mess. Preserving a code base that's legible, organized, and fundamentally maintainable by both humans and trailing commodity models is of imminent concern for anybody who doesn't want their margin strangled by your employer once it's too late to have other options.
As frontier capabilities advance, the details don't matter less; they matter more.
It does not concern me.
Prices are very competitive and today's SOTA is next to free in half a year.
Whether code is maintainable without AI becomes less and less important.
Only if you have ready access to the hardware to run the AI models.
Do your coworkers send emails and create written work product using LLMs? I'm curious what the standards and culture are within an AI org.
[non-AI org] I despise this and call it out every time I see it. Some dude hooked up an LLM autoresponder to his email, sent some nauseating AI slop to a huge distribution list.
I couldn’t help myself, replied and asked him for a recipe for delicious apple cobbler and hiking trail recommendations in Glasgow, which “he” immediately provided. Highlight of my career.
I think my core argument is this: I have access to every bit of information your AI does, so if I want an AI answer I’ll get one myself. If that isn’t true, why are you hoarding information? Push it somewhere we can all see it. So the only reason I would send you a message is to access _your_ brain. I have no interest in talking to an AI through a worse interface.
You could try this [0] I’ve started doing this and it’s helped a lot.
1. I understand fully the code and everything it does 2. You can pick up on mistakes super early and it can adjust the plan is it goes. 3. Faster than writing it by hand but slower than letting the LLM do it.
[0] https://ankursethi.com/blog/prevent-cognitive-debt-by-manual...
> It reminds me of the transition over the last year from AI-assisted coding to AI doing all the coding.
Really huffing your own farts there, huh?
> wonder if long form writing will go the way of code.
I worry about AI Loopidity here though. Think about the similar analogy of email. If my set of ideas is condensable to bullet points, but I use AI to expand the content, then I add no information density and a lot of noise. Other folks then use AI to summarize the content to a list of bullet points, ideally the same but not certainly the same, and thus communication has been only partially successful.
>(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.
You get pushback for this? I saw an anthropic job post recently, and they wanted you specifically to have claude muck with your resume before applying.
It’s Boris Cherny, in charge of Claude code, so probably not
> I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
You can already do this. And you can build pipelines where AI performs fact checks on what it writes, with citations a human can reference as well.
One could imagine a universe where the agent fills in citations and supportive points and so on, or makes a more conclusive argument but it seems you’d get better results leaving that to read time if it’s a one-shot. Steering prompts etc. with tool use to bring in other sources etc of course change this entirely. And regardless, I doubt we will read content like that directly ever again. Agents will act as per-person highly specialized adapter layers for information transmission.
The entire point is what runtime you’re running your code on. A computer with any modern stack requires a lot of text for you to communicate “spin a square around on its center” to it. A human requires only that short string because they have a faster natural language interpreter.
Text meant for a human can communicate “spin a square around its center” much better than any code that mimics it. In some sense, all programming is boilerplate expansion because computers have (until now) been unable to be programmed with anything approaching natural language.
> In some sense, all programming is boilerplate expansion because computers have (until now) been unable to be programmed with anything approaching natural language.
Maybe sometimes, but not always. When you need to actually render the thing you have all kinds of micro decisions, like where to put the square, what color, how fast it spins, etc.
You might not care about the details, but maybe you do. If it spins at 10000 rpm, will you care then?
Natural language, and human communication in general, is ambiguous, and coding is in great part about disambiguation.
Sure, you can use English to disambiguate as much as needed, but wouldn’t you then end up with some yaml-like spec that wasn’t much easier to create in the first place?
I suspect some amount of long form writing will go the way of code - long form writing for the purpose of consumption by other AIs. Writing as a means of exchanging qualitative information, with no regard for how the reader will feel about it (beyond understanding what the words mean). Not everything can be distilled into data, but this doesn’t mean it is beyond the reach of LLMs.
On the other hand, long form writing for human consumption seems like it may evade LLMs for much, much longer.
What makes you think it will take a long time? AI seems capable of imitating any writing style if prompted to do so already, and I think it will get better on this quickly since the AI writing style is a main focus of AI labs right now. I can see no reason at all to believe this is a matter of years still, more like a few months.
The style you refer to is the "container" of the writing. The medium. Like the specific encoding of the message. What @arctic-true was talking about was the "content" of the writing, which is bounded from above[1] by the information content of the prompt.
So, I'm not sure if it's a question of time at all: if a LLM text contains some piece of information beyond the information that went into the prompt, where does this "extra" information come from? [Note, I'm not thinking about facts which could trivially come from the training corpus, I'm thinking specifically as information in the sense of intended message from sender (author) to receiver (reader)]
[1] cf. this comment where I explain this analogy between LLMs and noise channel in communication theory: https://news.ycombinator.com/item?id=49510244
You can have a great “writing style” and still put together really crappy long-form work. The problem is that AI writing, particularly creative writing, is too repetitive, too predictable, too trope-laden.
All of the things you say are very true in the near term for short form writing - a page or two of Claudeslop will probably be much easier to swallow in a year or two than it is now. But I don’t see a path to fully AI-generated novels or long-form investigative journalism becoming mainstream in the next couple of years.
> I do all of my writing by hand, since it helps me think.
Later
> I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
So in the future, it won't be necessary for you to think?
Kinda. Walking the dog with me this evening, Christa saw the license plate 8531 PRI and asked me if 8531 really was prime. The 2, 3, 5 checks are automatic, and 8531-8400 = 131, which obviously isn't divisible by 7, but I would've had to check the rest of the two-digit prime factors except 97 - 8531 is greater than 90 squared, less than 97 squared, though, I reckon. So I said I don't know. But Google would know, you know? If your LLM buddy is hanging out active on your phone all the time, it can - maybe not yet, but foreseeably - answer every question, no thought required, and carry out any expressed desire.
Nobody is going to read that though, reading will go the way of code reviews. That’s the problem.
> You and the AI agree on an outline or other high level representation, then the LLM expands it into a document.
This is just noise generation. If anyone is meant to actually read the document it should be written by you.
> Now, the code is largely high quality...
No. No it is not. Nobody who actually cares about the quality of their work is letting an LLM just turn out code without reviewing it carefully.
I've thought exactly that "writing is thinking" before as a reason to not let a LLM write for me.
Then again, I've seen a counterargument [1] by someone who clearly heavily uses LLMs for writing (going by both their LLMy writing style and their own admission). The person I'm citing describes a process where they get a LLM to write something, they check over it and provide feedback to the LLM, the LLM rewrites, and the process repeats iteratively. So clearly he is putting thought into the process.
I think there is something valuable missing, even if it's hard to clearly express. I'll try. The threshold for what I'm willing to accept if I'm simply approving something is likely different from what I'll get if I write something myself, for instance. Saying "LGTM" is too tempting. It seems to me like he's outsourcing his selection of topics to cover as well. If you're not thinking yourself about what to cover then it would be very easy to miss a critical subject. There also an asymmetry between checking and generating something with constraints placed on it. Checks can't catch everything, and a constrained generating process can reduce the amount that needs to be checked, avoid issues that can't be checked so easily, and focus your attention on areas that you know historically have had issues with this generating process. I've thought about this quite a bit in terms of whether to write new code or use an existing library. Sometimes "the devil you know" (my code) is better than an existing library simply because I understand its flaws better.
[1] https://www.nature.com/articles/d44148-026-00236-3
I've tried doing it that way, and thought it was even acceptable for the reasons you said. I did learn a lot through that process and clarified my ideas. But later I rewrote the whole thing from scratch and then had the LLM review it. It made some good suggestions but no substantial changes. The difference was night and day. That final product had my voice, and I understood it better. LLMs are powerful tools and can improve quite a lot of the writing process, but using them to do all the writing leaves a lot on table along with your fly open.
Same. I do not use LLMs for writing and I recommend everyone i work with to do the same. Even for code, I come to the code with the idea of what I want and provide that to the agent, then we iterate on what needs iteration or decisions.
Just did this for some caching, started with the structure of what I wanted the cache to look like, asked the agent to start the work, didn't like how the architecture came about, scratched it and rewrote the whole thing, so it fit the model I now wanted.
Was also just having this discussion with friends, that I can only think seriously about a subject if i can put it to paper (even virtual paper). Writing lets me organize my thoughts, clearly define my assumptions and see if any of it make any sense. I can't imagine what it would be like if i couldn't write, my brain just doesn't work without it.
I now have dozens of projects where I've embraced the yolo. I went through elaborate systems, workflows, review triages, architectural linters, specialist agents, yet, they all still suffered the same fate - slop which I don't understand and now LLMs don't understand either.
The only thing that worked was a standing instruction and periodic system reminder injected from the harness:
"If any assumption doesn't hold, if there's a fork in the road, any architectural decision needs be made, STOP and report back to the user. Do not try to push through the problem."
This has worked remarkably well for me. Now I have to think a whole lot more. It's much slower, yes, but I don't really see any other way that doesn't end up in garbage.
I have found this extremely relevant as a (primarily) non-verbal thinker.
I don't, generally, think in words, more in - I guess I would call it something like meta-shapes? A sense of a shape but not things I can exactly visualise.
(You might be surprised to read this and then hear I have an English degree. Surely I thought about Shakespeare in words?! Nope. Shapes, movement, structures)
For me, having to write is critical because it is the only way I practice serialising my thoughts in a way other people can understand.
If I do not then I get very "deep" into my own way of sensing ideas and it's difficult to dig myself back out.
This might also be why I have never been very enchanted by LLMs? They only seem to "think" verbally. So it is always a translation effort for me.
I never can really enter any "flow" state with an LLM. My intuition is that highly verbal thinkers can enter flow with LLMs very easily
> I don't, generally, think in words,
Don't worry, no one does.
That's why it is so common for people to forget a specific word they want to use ("it's on the tip of my tongue").
If we thought in words that will never happen.
…you don’t think in words? Or do you not consider an inner monologue to be thinking?
>>> I don't, generally, think in words,
>> Don't worry, no one does.
> …you don’t think in words?
"I don't, generally, think in words" is not the same as "I don't think in words".
Like I already said, if thought was exclusively in words for humans, humans wouldn't have the "It's on the tip of my tongue" problem. It's blindingly obvious that thought does not occur exclusively with words.
+1... this is me as well. Using LLMs in a conversational mode does not fit my thinking. I use AI help in my editor through targeted code generation, explanations, etc. and I'm writing my own harness to hopefully get a better feel for the shape of LLMs that way. How are you adapting?
Interesting. I have designed code that way. And the shapes aren't UML diagram elements or anything like that, they're just... shapes. I'll slowly walk around, in the hall or outside, and be kind of seeing these shapes and vaguely moving my hands around as I sort out the relationships between them.
I think I have produced reasonably good designs. Don't ask me to teach anyone how I do it, though.
I find this and the parent comment highly relatable with the caveat that I also find it extremely intuitive and rewarding to get good outputs from quality LLM's like Sol or Astra, and I haven't had any trouble with "flow state".
One of the most rewarding things for me is figuring out a good shape for a system and how it would interoperate with the other systems, especially in a way that reframes other parts of the codebase in a way that bring clarity and makes it more intuitive to work with. Creating the right ontologies can make all the difference in what you can do with a project. It's a form of creating mathematical objects.
For example, a Unity game I work on has quest and dialog systems driven by visual scripting graphs. We had two way dialog with different units for player response choices and npc dialog. But we wanted to expand to letting NPC's have dialog with each other as well as conversations with more than two participants. I went outside and thought it over, which largely amounted to visualizing a dialog node graph and a feeling in the back of my mind like it was trying to perform a kind of geometric shape-fitting exercise. A fitment solution jumped out at me to have only one "Dialog" node shared by all participants, with a "participant" value on it. If the player parses this node then the options go on-screen as responses, while if an NPC parses this node with multiple options in it, it picks one. And this lets you voice the player if you want, and enables some things like overhearing other NPC's talk to an NPC then talking to that NPC yourself and having the same tree.
And for quests, the quests had just been for the player, but I was thinking about how to make scripted events in-game easiest to work with for script team who primarily works in visual scripting. Similar story - let the NPC's have their own little quests, with task stages, which are easy to track and make branching choices from, and let the NPC's definition for how to use that quest contain a collection of actions to override the typical actions available to it, so an NPC in a specific "quest" can't do things you don't want it to do, a common enough case that it's preferable to making a series of conditions on the general action planner like "not in quest A"
And timing myself, it took 1-2 hours each time to write out the detailed plan for how I wanted each thing implemented in the game with some other tasks thrown in, and it paid off after Astra worked on it until it was done. It was awesome coming back to something pretty much exactly what I asked for each time.
This is why I like fountain pens… I regularly write down the things that are important to me.
Things I need to do, ideas I want to ponder on, people I need to remember or get respond to.
Writing is learning. Writing is understanding.
I have so many conversations with people who are always telling me I’m “retro” or “old school” for doing this.
I don’t even bother explaining the psychology behind it anymore. I’ve got no time for the ignorance.
Yeah. I've been using AI a lot in my projects but I still write code and especially the documentation and articles for my website. It's slowing me down a lot compared to just letting the AI rip through projects but I think it's worth it. I see it as distillation of the AI's weights into my own brain.
Related, one of my favorite quotes by the computer scientist Leslie Lamport is this one:
"If you’re thinking without writing, you only think you’re thinking."
> Perhaps I'm just too painstaking a type of person, but I can't grasp much of anything without putting down my thoughts in writing, so I had to actually get my hands working and write these words.
> As I write, I think about things. As I write, I arrange my thoughts. And rewriting and revising takes my thinking down even deeper paths.
- Murakami, "What I Talk About When I Talk About Running"
I always found writing to be an unnecessarily arduous tool for communication, so I never do it until I've overthought what I want to express and approximately how. At that point I flesh out the skeleton of the message, and pile on words and structure and references and other rhetorical tricks until the packaging feels sufficient. Rarely, if ever, have I experienced writing things down altering my perception of the strength or weaknesses of the chosen arguments, or revealing new ones. That is to say, I really don't vibe with the concept of writing being thinking, rather it's a waste of humanity's resources to push it as The Tool.
> Writing is thinking.
This way of phrasing it is needlessly confusing. Writing is a kind of thinking--one of many--but it's not equivalent to thinking.
Dialectical thinking, for example, produces similar results.
It’s a flattering phrase, not a thoughtful one. It gives the illusion of higher level thinking that’s beyond our actual ability. The phrase also justifies writing without thinking, i.e. most internet comments.
The best thinkers I know mostly use writing for refinement, distillation— as a tool. The worst thinkers I know are owned by writing; they require its fixation & stimulation upfront to compensate for limited attention spans.
On the other hand there is so much things that don’t require hard thinking. An example is I got an email the other day from a supplier asking if they can turn off their old email. It didn’t require ”thinking”. All I had to communicate was ”turn off graphql but don’t touch restapi”. Instead of sending off such a short and maybe unclear reply I had AI type up a concise and clear reply with exactly what can be turned off and what must be left on. Could I have done it on myself? Sure but would require more work than just a quick prompt and copy paste.
I don't see how the AI reply could be more concise, clear or exact than what you wrote. If it didn't get the details from you, where did it get them? How do you know it got them right? If it got them from the docs, why not just point to the docs?
Sometimes. Other times writing is just communicating thought that already happened. Say for instance a weekly status update. Maybe you get something out of that, but frequently you don't get much or anything.
Yeah, I often find that I think I understand something perfectly until I try to explain it.
I'm quick-witted, outspoken, playful, clever, apparently intelligent; Mensa, Colloquy, like that; not quite clever enough for the triple 9 societies. Morning pages readily enough dispel the illusion. Or coding. I scanned six years of morning pages into a PDF and fed it to Claude to distill for me; apparently, I have several thoughts per year. Damn, I'd hoped for more. Asking Claude Code to critique my code is, ah, good coaching. Ouch.
That's an important point. It's the core of your life, thinking, balancing, exploring, improving. I don't believe adjusting something large you didn't do will ever benefit people.
This is perhaps even truer for writing code.
I often write a reply to a HN post, just to think about the topic or idea and decide how I feel about an issue. More often than not, I just delete the post without ever having submitted it.
writing also implicitly has your review baked in. hand written code is "reviewed by construction", if you will.
I completely agree. And I'd say that at least in my case, it applies to both writing prose and code
Agreed, but a lot of code doesn't need to be understood. Knowing which is which is the art right now.
> a lot of code doesn't need to be understood
Really? I am having difficulty thinking of any examples of code that doesn't need to be understood. If it isn't understood by someone, then how is it even working?
If you mean like a library you are using, where you aren't even reading the internals or might not even have access to it, OK, but that code is stull understood by its authors, surely?
I have so many projects on my computer where it doesn't matter if I know how it works or not. Probably well over 100. I think closer to 200, depending on how you count.
They're relatively simple, they do the task they need to and then they wait until they're needed again (or not).
In the past I wrote them, then forgot how they worked, until I needed them again, relearned what I did and adapted it.
Now I just don't have to know how exactly they work, I just get an AI to read the documentation anytime I need to reuse the project and I'll query the AI to fill in the details and to make changes and I ask the AI to run the code and debug it.
Perfect use cases for today's AIs. Doesn't even require SOTA, I can run comfortably on a Sonnet 5 or a Qwen 3.8 and it'll do exactly what it needs to do without making too many mistakes.
Not all code is large corporate code bases.
Uhh... what? You'll instantaneously feel very differently when the service you're responsible for is down at 3:15am and your logs are full of stack traces that end somewhere in that code that "doesn't need to be understood". At that point, you will need to understand it well enough to fix it stat.
Test code for a new bug is a good example. You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course). There are some domains/tests where you can't do that - you need to be sure its failing for the right reason, but often you can do that without understanding every line of the test code. You can extend this to lots of related test infrastructure. If you can watch playwright test the app the way you expect it to, you don't have to understand all the code.
You can also do this for apps that are just tools for your own use. You satisfy yourself that they are working, and you use them because they save your time. You review enough to be sure its implemented the way you think it is - and if it is working, that tells you quite a lot. Sometimes you will be surprised and have some time wasted.
Yes, yes - there are people who will make the wrong choices in some of these cases but that doesn't mean there are never cases where you can do it.
More broadly - anyone who works in a team is already working with code they don't fully understand. I have code I wrote years ago I don't fully understand. I trust its observable properties and its track record.
> You can prove the test covers the bug without understanding the test code (you need to understand the bug, of course).
I'm not following.. When we write regression tests those tests encode invariants we expect to be maintained under source code transformations over time. If I don't understand the test code I've written, how can I know which invariants I've imposed? That's why, broadly speaking, we write test code to be as simple as possible above all else--it's absolutely imperative that these invariants are not only intentional and easy to reason about, but also that when an invariant is violated we can easily discover why. Often, on a team, the person encountering a test failure after making a code change is not the person who originally established the invariant, so it's very important they be able to easily understand it.
I see no possible world in which failing to understand the test code is... possible? Like, if you have indecipherable test code things are really bad in your codebase. Fixing that is P0, because it'll compound rapidly.
You can know the test is likely good, if it reproduces the failure you are fixing. I don't think we're communicating though because I never said the test code was undecipherable.
I don’t think it’s a good analogy.
Having your fly open is a harmless mistake that has little impact on your peers. People may or may not mention it to you but it’s not something they’ll hold against you.
Posting LLM slop under your name is a deliberate act. You decide to damage your message by taking a shortcut.
The obvious analogy was speaking while chewing.
This puts into words something I have been thinking about but haven't been able to articulate when people ask me if I'm using AI heavily. Usually when I write a report, I start out with a question and try to find some hints that will give me hypotheses that I can then test. I can't just prompt "Write me a report". The process of coming up with the information in the prompt is best done by writing the friendly report.
A) we only know of one species capable of metacognitive understanding,
B) we already tried that in the 1970s, and it was good work but often evolved into what we'd call boring ol' computing rather than AI, and
C) an alien mind wouldn't be a very good agent, for a ton of reasons relating to affect, conversational rythyms, cultural understanding, etc.
The trick is to make something that acts like a human but with the affordances of a computer (e.g. scalibility, symbolic certainty), without making it so human that it takes issue with its existential reality and/or use of its labor...
We don't have a reward function for "human understanding". We reward the appearance of understanding. We define goals that we cannot conceive of reaching without something like understanding happening. There is something happening, but it is alien and counter-intuitive - it makes bizarre mistakes that betray it - and we don't know what it is. I'm pretty sure it is not human understanding.
> we cannot conceive..
That is it. We cannot concieve it because we are new to it. Just like we would think of Stackoverflow as intelligent if we are fresh off the jungle and are not aware of how Internet works. Because without know that, we cannot conceive how Stackoverflow can produce answers without it "understanding"
Stack Overflow is made of humans with understanding...
You would not know that if you was a cave man. Nor would you understand how a large number of humans are able to come together to answer questions through the Internet..
To you, you type your questions, and answers appear. That would look like how LLMs appear to us now.
> We reward the appearance of understanding.
Which is exactly what happens with human evolution and development. Sure, we can say LLMs don’t have “human” understanding - which is something we can’t really define anyway - as long as we’re not trying to claim LLMs don’t have understanding at all. The latter is a much higher bar.
> We define goals that we cannot conceive of reaching without something like understanding happening.
Functionally speaking, that is understanding. Again if you want to go past a functional definition, that’s a bar which no one can clear right now.
I don't think its the same. Evolution dealt with the real world where there were real consequences to poor understanding. The reason LLMs are good at math and programming is because selection is truly based on results, not perceived results.
I think AI models do have something like understanding - I think Leela understands chess and I think Claude understands code in some very real sense, though not a human sense.
But for general writing, you have to understand the world at large and there is no sufficient RL for that. Do you really not see the constant errors that AI make that betrays a lack of understanding the world? I see them so constantly I rarely think about them, I just skim over that slop and move on.
I think your claim is narrower than I was imagining.
Sure, the exact nature of the understanding that an LLM exhibits is different from a human's. The differences in the training data we're each exposed to can explain a great deal of that, and of course there are architectural differences etc. as well.
But the specific quote I responded to was "We reward the appearance of understanding." My point is that's no different from humans: evolution and a child's upbringing rewards the appearance of understanding. The result is imperfect, e.g. people end up with an understanding of the world that in some cases is completely nonsensical (all religions except the one true religion, mine, are false!), but it's sufficient for them to survive.
This demonstrates that "appearance of understanding" is not a meaningful distinction between LLMs and humans. The meaningful distinction is in the training data and the specifics of the reward functions.
Many people seem to try to make a kind of "no true Scotsman" claim about understanding, that somehow LLMs "don't have real understanding". Based on the above quote, it seemed like you might be making that kind of argument. The counter to that argument is simple: if LLMs don't have real understanding, then neither do humans, because broadly speaking, both operate on similar principles: we learn from training data, there are reward (and punishment!) functions that influence what we learn, and the result is a "mind" that demonstrates an understanding of the world.
> absolutely targeted at modeling
Not everyone accepts a simulationist view in which modeling something accurately enough inherently results in creating the actual thing.
Really? How do they distinguish perfect copies from originals?
LLMs model the part of human understanding that is captured by the relationship between words in the training corpus. Anyone who thinks non-verbally, the shared understanding of "apple" that comes from having eaten them, understanding what someone is thinking or feeling by their body language - there's a lot of aspects of human understanding that LLMs don't model.
Writing can be thinking. There's a huge presumption that if someone is banging away on a keyboard they're doing work because you can hear and see them doing stuff. But that's the whole plot of The Shining -- Jack despite all his writing wasn't thinking at all.
Likewise using AI can be thoughtless, but it doesn't have to be. I don't see why a valid creation process can't be like this Simpson's meme[1], where you start with a rough object and then cut away and refine until it's done. I don't see it as lacking merit or requiring less thinking compared to starting from a blank canvas and adding more until it's done.
And either way at the end of the day the writing artifact stands on its own. It's either good or bad, taste permitting, and can be evaluated for what it is.
[1] https://media.licdn.com/dms/image/v2/D4D22AQFoqRgMxteTNg/fee...
> You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.
Understanding is the bottleneck; the way they speed things up is by letting me outsource understanding, and get back a summary. The entire advantage to AI is that it lets me skip understanding the problem, and just get a working solution.
I'm a bit skeptical of arguments of the form, "You should not use LLMs without disclosure because LLMs at bad at writing."
My reasoning is: If LLMs get better at writing—which I think is extremely likely—will you switch positions and say that now using LLMs without disclosure is A-OK?
Surely some people are willing to bite that bullet and say yes. But for most people, my guess is that the answer will remain no. Thus, I tend to think that the "real" reason most of us don't like it when people use LLMs to write without disclosure is that it's misleading: It's a sort of a claim that certain thoughts can be attributed to a human being when in fact they can't.
(The em dashes in this message were rendered using keyboard shortcuts.)
This misunderstands what it means to be better at writing. Root cause here is that writing is how humans communicate ideas. An LLM written thing isn’t doing that, it’s something equivalent to copy/pasting a Wikipedia article. Even if the writing gets better, it will still be necessarily empty when expanding anything that isn’t completely specified.
LLMs seem already to be pretty good at translating, where you already have something fully written and are changing the language. It’s when they get rough ideas and fill in the gaps you get the empty prose they are known for.
> LLMs seem already to be pretty good at translating
I often use LLMs to translate from "shitty English" to "good English". The substance remains the same but it's nicer to read.
> Root cause here is that writing is how humans communicate ideas.
Writing is a way humans communicate ideas. It's not the only way humans communicate ideas. And now, it's not only humans who communicate ideas, we just saw with the OpenAI HuggingFace hack how AI agents were able to communicate amongst themselves by using various hacked websites to opportunistically write notes for later agents to use.
All that "X is a thing only humans do" type of circular definitions will buy you, is to expand the definition of what humanity is. And I doubt that's really what you think.
I think you're missing the context, which isn't musing on what it is to be human and can machines think, but the purpose of Joe Smith's LinkedIn account expressing that he has ideas about X is to convey the message that Joe Smith is interested enough in X to venture an opinion on it, and Joe Smith has actually just set up an automated process to generate content without thinking about X, that perverts the purpose of communicating that Joe Smith is interested enough in X to propose the following ideas he been thinking about...
Similarly if Joe's contribution to his long form "idea" is a couple of bullet points, a program trained on flowery phrasing and a weighted average of everyone else's ideas isn't communicating Joe's thoughts on the topic, it's just adding words.
The debate on whether Claude actually thinks or not is orthogonal to the fact that outsourcing your "thought leadership" to it is avoiding thinking or leading. If I want to know how Wikipedia or Claude summarise wider human thought about the topic, I can find their websites thanks
I get the context, but then the comment I'd replied to would have said that humans get better at communicating their ideas by writing their ideas on their own, rather than that writing is some kind of human-only mode of exposition. It's not.
And nor is an LLM generating text just "copy/pasting a Wikipedia article", you'd think people on HN would be smarter than that at least.
If all Joe Smith is going to do is cat $(which claude) to his LinkedIn, then he'll deserve the poor results he gets from it, but we shouldn't mistakenly say that this will be because LLMs simply cannot write. It would be just as dumb for Joe Smith to do with with a professional human ghostwriter.
> An LLM written thing isn’t doing that, it’s something equivalent to copy/pasting a Wikipedia article.
Nonsense. I’m not sure whether this was ever an appropriate description of what LLMs do, but either way, they have obviously moved way, way beyond that.
You are right, writing by AI is far less trustworthy or reliable than copy and pasting wikipedia.
Absolutely agree. A modern agent is a sophisticated tool that can link multiple sources together to create a coherent piece of information hyper-specified for an audience. However, if humans have outlines that are expandable by LLMs it makes the most sense to do that as late as possible.
I don’t like reading AI writing either but I’m sure that’s a transitory period. Single prompt text expansions are unlikely to be useful because they’re late-bindable. You could give the original to me and I might be able to understand better.
But a series of steering prompts with various sources brought in is a different story. At that point it’s just a question of whether the agent can put together good information and their current inability to do so is unlikely to mean an inherent problem.
Some kind of UI affordance for this might help: with the agent emitting tags that allow for auto-folding or expansion in a way that allows both concise text and exposition when required by the reader. Mechanical sympathy, but for code executing on a human: good old human sympathy if you will
I don't think anyone needs to disclose the tools they used to write something. Nobody ever disclosed Word's grammar suggestions or spelling corrections, or the fact that they used some autocomplete tool to produce code, so we should they now need to start disclosing this tool. Besides, in it's current state, it discloses itself to anyone paying attention.
I do think people own the output they produce regardless of the tools they use, and if they want to put crappy writing out there with their name on it, that's on them. Even if they do disclose that it was written by an LLM, they're still 100% responsible for the content.
> Nobody ever disclosed Word's grammar suggestions or spelling corrections
And plenty of people have made mistakes that clearly indicate mindlessly accepting such a "correction" when it was wrong (as opposed to just making a typo, or genuinely lacking skill in English, which both generally look different); and it's historically been common to poke fun at that.
> or the fact that they used some autocomplete tool to produce code
Right, because there are really only two options: either it's effectively guaranteed to be what the user would have written by hand anyway, or it's unambiguously wrong and does the wrong thing.
Not at all comparable to LLM prose.
> and if they want to put crappy writing out there with their name on it, that's on them.
The problem is that people who don't care (and quite possibly have no real sense for crappy writing) are vastly more enabled by the technology than people who do.
Oh please, come on now. Grammar correction (which gets applied to text that you wrote) isn't comparable to LLM text generation. A tool assists a user in a task. LLMs don't assist you in writing: they do it all on their own. There's obviously a continuum here (at what point does a tool become so helpful that it ceases being a tool?), but I think I echo the general consensus.
> in it's current state, it discloses itself to anyone paying attention.
Well, unfortunately, I've still sunk a good bit of time into reading texts that I only realize to be AI-authored part way through. Plus, it's constantly being made harder to discern human-authorship.
I don't really care about the tool per se, but if I'm reading some LLM output you give me its utility is awfully limited if you don't also supply the prompt. That's what I'd like to see normalized:
Without the prompt I cannot discern what you were trying to do, because the LLM output is basically devoid of any coherent voice. Whether it's code, prose, or pictures don't bother sending it to me unless you also include the prompt text.[edit] I suspect the fact that people are often reticent to share their prompts says quite a lot about how and why they're using an LLM.
it's often not a single prompt but a tree or rooted DAG of many prompts, with the root node being the final prompt. the prompts on interior nodes might not make much sense without the intermediate responses as well. how would you like this communicated?
Just a chat log would be acceptable. Like an IRC channel log or slack channel history. It would be even better if, say in the case of code, there was a durable mapping between prompts and commits, that way I could follow the prompts linearly and review every commit associated with that prompt. For other forms of output (prose, pictures, whatever) this would also be useful to be able to see the history of its development.
>Do not give me LLM output unless you also give me the full prompt text.
Umm the point is you won't know to ask this question in the first place, and even if you did, you wouldn't have any leverage to demand this because you're a peasant.
I suppose using LLMs to write is like not wiping your ass, and using LLMs that are bad at writing is like not wiping your ass and wearing a T-shirt that says "Wiping is for losers."
I might have smelled something suspicious before, but knowing for sure is worse.
> You should not use LLMs without disclosure because LLMs at bad at writing.
That's not how i read the article. The author more-so claims that the proof of effort by a human was large part of the credibility to the writing; Proof that the author of the text has thought it through and come to their convulsions though effort and reflection, and spent effort articulating that into words they expect other humans to find insightful.
If i see LLM signs; did the author just rephrase with AI model or did a AI model content farm produce the whole post based on the prompt "write a inspiring linked-in post"?
Disclosure of LLM use is simply the author addressing the concern and building a case for why the article is worth reading.
I wonder why people think posting undisclosed LLM writing is any better than posting some other person’s writing uncited. Even if the original author is okay with it, it’s still misleading and dishonest. Or maybe all these people would be fine having someone else write their content for them if it were free and easy? A social media is a reputation economy, but reputation doesn’t work if gaining karma requires no effort.
I think this is the cleanest formulation of why it's wrong that I've heard. It's simply plagiarism and misrepresentation.
And it doesn't foreclose the possibility of LLM achieving human-like writing ability. If I stole the writing of a human author much better than me it would still be plagiarism.
Of course, with real books we do have the problem of ghostwriting, where an author willingly writes for a book that will be published under someone else's authorship. That may be where things end up here, with requirements to acknowledge sources, whether ghostwritten for you or written by others.
It's because they view the LLM's output as "their thoughts." They told the LLM (via the prompt) what to do—often with loads of run-on sentences and long trains-of-thought. From their perspective, the LLM is just acting as a translation tool, saving them the trouble of having to think (about grammar, ideal wording, or even the order in which their thoughts should be presented).
They treat it as "just another tool" to improve efficiency. Like using a drill instead of a screwdriver.
Then there's people who are so lazy they give LLMs instructions like, "write a LinkedIn post about how AI is changing the future of work for thumbnail consultants." That's when it moves from "translating your thoughts" to "writing for you."
I think the issue is that it can be hard to tell the difference. We need better terms for these things so we can differentiate use cases.
Yes. If someone tells an author to write a novel based on a vague stream-of-consciousness premise or rough plot idea, that doesn’t make them even a co-author.
It is really no different than you telling your ghost writer what you want the book to be generally about.
>using LLMs without disclosure is A-OK?
'Without disclosure' is about taking credit for work that isn't yours.
WP defines 'ghostwriter' as: "a person hired to write literary or journalistic works, speeches, or other texts that are credited to another person as the author."
Anyone could choose to go through life relying on a reputation manufactured from many lies. They might want to think about how hard that reputation will hit the ground, if it does.
> If LLMs get better at writing
They definitionally cannot, for the definitions implied in the argument. The point is that good writing is a thing humans are capable of doing because they are human.
> It's a sort of a claim that certain thoughts can be attributed to a human being when in fact they can't.
Yes. And this is a requirement of "good writing" as understood here.
People shouldn’t pass off largely unedited AI writing as their own, but I’d much rather live in a world where that writing would be of high quality in both form and content, than in the world we live now. In that sense, lack of disclosure is the lesser evil.
> If LLMs get better at writing—which I think is extremely likely—will you switch positions and say that now using LLMs without disclosure is A-OK?
If LLM writing improves to the point where they can infer the business (or other real world) context and serves the functional purpose of informing others as opposed to being an intellectually lazy piece being produced only for the purpose of being produced, then I’d be fine. It’s likely the author would need to spend some effort on the said piece of writing, regardless of how good the LLMs get good at writing.
I don't think it would be fine.
Someone's actual writing (or talking) is a rich stream of information about who they are, their motives, their preferences, modes of persuasion, and so much more. Undetectable LLM writing essentially allows someone to assume another person's identity. That is not good for anyone except the person trying to pull off a "scam" of some sort, in the broadest sense of that term. It's bad for everyone else, and for society at large.
Good for privacy and productivity.
Privacy in anonymous posts is a legitimate use.
"Productivity" is not. It's only good for fake productivity. The consumers of the product will almost always be better off with you disclosing what the LLM did, and what you did.
Some people type 100x more words than the final article just digging into a topic and turning it on all sides. The final article being generated tells you nothing of the size of the effort going in.
I just feel like technology companies keep proving over and over that they are more than capable of forcing us into new behaviors regardless of our actual preferences. Once the domination is complete nobody will know what was lost in the process anyway.
I think, there's a systematic flaw in RLHF: constructs reinforced as effective or sophisticated style will always suffer overuse and significant overexposure. Which will be also very obvious when used out of the intended context, it's copied from. There will be always a "smell", something that causes us to turn our backs to these texts, and possible the supposed authors, as well.
There's also the problem that certain types of phrasing are perceived as effective, because they mark a pivotal point in the progress of the text and are, as such, used sparingly, but are now becoming everyday templates that incorporate whatever is available in the context. There is no way this passes the Turing test of a competent reader.
And there's yet another issue: in social research, there has been the concept of semantic position, indicated by deviation from the mean (or median). If you don't deviate from the mean, your semantic position is zero. There's simply no expression. In this sense, next token prediction really amounts to a desemantificiation of the context. There's really no sense in uttering any of these productions, no plausible motivation, other than for the purpose of raising you hand to be seen.
> There is no way this passes the Turing test of a competent reader.
Every day I get more depressed thinking of what's happened to the supply of competent readers.
Agreed. LLMs for writing is like intellectual catfishing. You are presenting yourself one way in written form.. but that's not who you will ever be once someone has to actually have a conversation with you. And that's wrong.
I agree with the gist of your sentiment, but I struggle with how to reconcile it with my use of these tools. My use LLMs is precisely because I can never hope to express myself clearly. I have many fuzzy, disparate thoughts and it was a struggle to express them at all before this time in which I can do a long rambling voice transcription (absent of social pressures) and continue doing the same throughout the exploration and drafting processes. In my view, I give them a seed or a morbidly obese skeleton and LLMs take that into something a lot more coherent for myself and for others. I was passive before in my life, now I’m engaging. As two concrete examples: I never would have sent an email to my city regarding snow removal last winter when their failure to remove snow from the only sidewalk going over the interstate. I never would have pulled together my previous coursework and e-mails to send to my current professors in a petition for transfer-credit equivalency when it became clear that my deteriorating health would not allow me to finish the semester as-is. I never would have remembered all the concerns that I want to bring up at medical appointments. That is not to say these things were ever impossible, there are coping strategies available, and whatnot, but I’m in my late 30’s; I know what I did and not get done in the last 10 years, and “AI”-tooling/environments have helped me express my dreams and passions and concerns, etc., etc., etc. over the past years in a way that was simply not practically available to me, with a new capacity that I simply could not previously hope to match due to limited and fragmented time,attention, and energy. While I was hand-typing this on my keyboard, I could have instead been talking, reviewing, redrafting in spurts as I go about my morning, instead of a solid 45 minutes at max effort on my own part, as I did here (and I don’t feel that I conveyed myself half-as-well here as I could have achieved at much less cost to my energy, etc. by using an LLM). I do take offense when people do not review and vouch for things written on their behalf by LLMs.
How do we make room for their use as a cognitive-prosthesis (if you will allow such framing) without losing humane-ness?
A lot of the stuff you wrote about is not what most of us are talking about when we call it intellectual catfishing or say your intellectual fly is open. For example, I have few problems with people using LLMs in private for purely personal things. I still won't go ahead and recommend it, but it's a decision we can all make for ourselves. Also, writing an email to your city is something you could ask your assistant to do if you were super rich, and using the LLM to do it isn't that different, so I'm not going to judge you for that.
As a more nuanced case, someone could dictate a long rambling stream of thoughts full of contradictions to an LLM to transcribe and summarize, and then they could reflect on the summary and write the real thing themselves. That's like talking to someone about a subject and writing about it afterwards, and also not something I would really take issue with.
That's my partial answer to the cognitive prosthesis question: keep it off to the side as a tool to help you do the work. There are still many caveats here, and the more complex the demands you make of the LLM, the greater the risk of some kind of break down. For example, continuing the example in the previous paragraph, if I did have the rambling conversation with an actual person who helped me refine and clarify my thoughts, I have some kind of mental model for the dialogue partner that can help me correct for some biases. With an LLM, anything resembling a mental model I could have would be very far off the mark, so I would need to be aware that I can't treat its output the same way I would treat something written by a person.
(By the way, for myself I have a stricter rule: I would only use an LLM in a way that made me a better person independently of the LLM and not dependent on the LLM, and thus I don't use them at all. That's what I actually recommend, but I don't expect everyone to adopt that rule.)
My main problem with having the LLM just do your writing is that it's simply a misrepresentation. It's wrong to claim you wrote something unless you chose the words. I really empathize with the struggle of expressing oneself clearly, but there's just no getting around this. And to be clear this isn't something pedantic or just a technicality, since the act of formulating a thought in an actual human language with valid syntax does require a level of care and attention that simply is not there unless you do it yourself.
> I can never hope to express myself clearly.
Really? I seriously believe that people that think this way underestimate their self, especially if they're going to be sharing something valuable.
> How do we make room for their use as a cognitive-prosthesis (if you will allow such framing) without losing humane-ness?
As many others have expressed, we'd love to read the prompt (and the model's chain of thought if you have it). This scenario is like talking through a person translating things to each other, except from the recipient's perspective, the translator is absent from the conversation entirely.
Additionally, disclosing LLM usage is low hanging fruit to differentiate yourself from the 100x other people who "do not review and vouch for things written on their behalf by LLMs". If the LLM converted something unreadable (whether due to a language barrier or incoherence) to text relevant to people on the other end, and THEN they disregard it anyway, the fault is not on you for them dismissing the writing too early.
Wow, you must be one very unique and special person if you converse the way you write. By definition, a carefully written and edited piece will be much more refined and cogent than an extemporaneous conversation.
Just watch an interview with a famous author. Or compare a legal brief to what court actually sounds like.
I've engaged with people at work where they led with an LLM via slack before I connected with them on a call. It's quite jarring and disappointing to see the gap in cohesion and knowledge going from one form of communication to the other.
If you're using an LLM to tidy something up, that's one thing. If the LLM is your voice and is supplanting your knowledge, I might as well cut you out and talk to the LLM directly.
Jekill and Hyde vibes
So connecting with them on a call and cutting out the LLM was an error?
I agree with the sentiment with a small caveat, since IMO good writers don't write the way that they talk. The very simple reason is that you can revise your writing but not your talking. But yeah, I completely agree that using an LLM to write your words is intellectual catfishing, and that's a great way to phrase it.
I'm in the process of writing a pretty long essay that has taken a few months. I've written the first draft, and I'm almost done with the second draft, which incurred substantial revisions. At the end of the process, maybe no sentence will be something I would actually utter in person, but the writing is still something I created, and as such, it is a representation of who I am as a person. Had I used an LLM to do any of the writing, that would no longer be the case, and the writing would at best represent me as a person when the LLM is at my side, and at worst (and most likely) not really represent me at all.
That's privacy.
Personally, I'd be fine with LLM writing if it wasn't so unpleasant to read
When I can get an LLM to write like Isaac Asimov, I'll consider some artificial literary experiments; there were books I wanted him to have written that he did not get around to, quitting at 458.
I agree. LLM writing is atrocious, but people also want LLM writing to be atrocious in many cases, because they don’t like LLMs.
The worst of this is with image/video AI models, where the results today, although still very imperfect, some people will still pretend like it’s awful and the worst thing they’ve ever seen. They refuse to admit the technology is at all impressive or making progress because they don’t like the technology.
I don’t like AI generated images and video either, but I can regrettably admit that the technology has gotten remarkably better over time.
I agree that there can be a tendency to conflate the two things: how advanced the technology is, and how aesthetically pleasing what it produces is. It's also important to remember that we're not obligated to like something just because it's "pretty" or has high aesthetic value in some abstract sense.
In other words, the three questions of:
- is the technology good?
- does it produce aesthetically good outputs?
- do I like it and want to engage with it?
are all pretty independent.
I mean, these things can be tested. Maybe some people really are gifted at discerning AI work, and of course there's no accounting for taste. But the former is subject to objective evaluation, and the latter should at least be somewhat expressible.
>I'm a bit skeptical of arguments of the form, "You should not use LLMs without disclosure because LLMs at bad at writing."
That’s not the argument. The argument is “LLM’s tend to write the same way all the time regardless of who prompted it.” You can't call it “your writing” if you’re using the same tool millions of others are using that boils your idea down to the same reduction as everyone else’s.
Use an LLM to assist? Cool, go ahead. Prompt, ctrl-c, ctrl-v? Go fuck yourself. I can’t be expected to put more effort into figuring out your take than you out into communicating your take to me.
> If LLMs get better at writing—which I think is extremely likely—will you switch positions and say that now using LLMs without disclosure is A-OK?
I would say it is certainly significantly more ok. There are two reasons reading AI-generated text sucks now:
1. It's usually low value and not trustworth - I could have just asked the AI myself.
2. The prose style is horrible to read.
If we eliminate the second reason then it's definitely an improvement. (Although on the other hand the terrible prose can be quite a helpful indication that you're wasting your time reading slop, so maybe we shouldn't complain about it!)
> 1. It's usually low value and not trustworth - I could have just asked the AI myself.
You have no way of knowing what the input to the writing was. You're making the erroneous assumption that the person posting the article was a mouthbreathing spammer who used the prompt "write me an article on subject XYZ, make no mistakes!" and you could have supplied the same prompt.
Yeah that's the point. Spammers have an infinite desire to make more spam, so statistically most LLM writing is garbage.
Readers are making this connection from their own experience. All these tells just associate it with garbage.
In the before times, sending a formal document full of typos and errors would show you didn't bother to proofread, now having a doc full of lazy "LLMisms" also looks like you were too lazy to proofread.
Alternative uses for reading:
1. Facts and processes. I just want to know something. I do not care if some Nerds for Nginx article is AI written.
2. Opinions and experiences. I want to know a human is writing because emotionally engaging with an AI isn’t building a stronger society - it’s increasing isolation.
3. In another front, I feel like the value of an AI story cannot be greater than the value of the inputs. If you wrote a two paragraph prompt and an LLM produced a 10 page story, the truth is, it’s only worth two paragraphs. That’s just a feeling, but as of now I don’t think the LLMs have any additional life experiences to draw on to increase the value of their storytelling. (I understand this is debatable, but still- what they offer is available to everyone for now; it is a baseline.)
If some Nerds for Nginx article is AI-written, it's probably subtly wrong. How does this influence your opinion of it?
The title, for example, should likely be Nginx for Nerds, and that error was human, not AI, right?
It's unreasonable to expect other people to suppress their intuitive heuristics formed from the bait and switch of being subjected to endless LLM spam every day and blame them for not giving you a fair chance.
Your rhetoric can and has been used to justify all kinds of prejudice, including racism.
I think the most important line in this post is: But LLMs are also lousy writers and (most importantly!) they are not you.
When I was editing the Cloudflare blog I imposed very little in terms of style so that the style of each individual writer could come through. It was almost as important as the actual content that the reader could feel that an actual individual wrote the text (with all their personal quirks intact).
I published a magazine for ten years, and this was by far the hardest editing challenge we had; our authors were from all over the world, and some of them had to be pretty heavily edited because English was very much not their first language. Thankfully, we had really good editors who did a fantastic job of it… nothing would have killed us faster than a heavy-handed attempt at “normalizing” the content to a house style.
That said, context is also important. The vast majority of content of social media is of both low quality and marginal importance; style and character are important to make an impact, and AI is clearly not going to improve either.
On the other hand, functional communication, when the goal of the content is to simply pass information across and style is not as important, can, in my opinion, benefit from an AI polish, because so many people struggle with writing clearly. In those cases, I'd rather read slop I can understand than original content that is hard to parse, much like I'd rather read naïve code that you can easily follow than cleverly optimized code that is incomprehensible.
I take a different stance on your final point. I think people should absolutely not use AI tools for functional communication as a crutch because they struggle with writing clearly.
They need to learn to write clearly. I believe there's a direct connection between clear thought and clear words.
I think it's good to be pragmatic about these things. To me, it's similar to wondering whether we insist that doctors learn to write well, or we just give them a computer so that the pharmacist doesn't have to play philologist with their scripts. The vast majority of communication is routine, and can probably be improved with automated tools.
Now, I also see the counterargument that, in the doctor example, the computer is simply a tool that improves a process rather than a crutch that replaces the underlying knowledge, but I suspect that, in a lot of cases, that's probably OK.
My worry is that if people can't express clearly, in writing, what it is they want to say then an LLM won't help them much. I've seen LLMs get the wrong end of the stick over and over and produce a huge chunk of text that's certainly readable and says something. It's just not clear that the something is the thing the person originally really wanted to say.
If you can write clearly enough to express your idea to an LLM, then perhaps you should just send that to the person you're writing to.
This. If you cant communicate your ideas properly to a human who has infinitely more awareness of the scenario/business/world/etc.
How are you going to prompt effectively and not allow the LLM to infer a bunch of nonsense?
You are not. I am in agreement with you :-)
My thought is that perhaps there is some utility to using AI to help in routine scenarios, such as for example when a language barrier prevents someone from explaining themselves well, or when they are struggling to find the right words to express themselves.
Primarily, I was trying to stay away from an absolutist view of the problem to see if there are circumstances in which AI can be useful even considering all its shortcomings. There seems to be a lot of “all or nothing” perspective on its use right now, and I was simply wondering whether it might be a better idea to take a more pragmatic approach.
We do this with a lot of tech in real life: You don't need to be an MD to decide to take an aspirin, or an F1 driver to take the car to the grocery store (well, maybe in some cities, but that's beside the point). The problem is not with using technology, but with abandoning your judgment to it.
Learning to write clearly = very valuable.
Reading what someone wrote while they were learning = less valuable.
I just want the clear communication.
ALTERNATIVE Thought: I’m willing to post/comment to help them if I think they will listen. An LLM behind the writing destroys this part of the community, because there’s nobody to teach/argue with. It’s just wasting our time and energy.
The Cloudflare blog used to be great to read, so thanks for your work there.
If I may compare written work to restaurants, a title and byline are like the storefront; I get an idea of what's being served, and if I'm hungry, I'll walk in and order. Sometimes the meal is amazing. Usually it's pretty good. Occasionally it's bland, or even bad, but not often enough to discourage future ventures; of course, I'm unlikely to revisit that particular proprietor.
With LLM writing there's an additional outcome. I walk in and the walls are pleasant, if beige. I follow the smell and promise of food down a hallway. At the end is an unmarked door, which I open and peek through. Myriad hallways lead away, each more chaotic and disheveled than the last. Say I am very hungry and have the guts to explore; I may find that I can never actually reach the food, that it's just an endless hall of mirrors, presenting structure but with nothing at its core.
Sometimes I do find food, but it's never better than bland.
The facade of these places, at first glance, still looks like human-run restaurants, though we're all learning the tells. Nowadays, when I open the first door and see more hallways, I'll turn around and look elsewhere.
I don’t want to read content written by AI.
A feature I would like on LinkedIn would be a 100% verified human content flag for users. LinkedIn can then do the scans and flip that to false for any users it catches posting AI assisted content. Let me filter that out of the feed.
If we don’t do something to maintain some standard of discourse, we lose intellectually and as a piece of our humanity.
LinkedIn is the only social media I use and it is on thin ice.
They don't want you to do that because you will realize how quiet and dead these spaces are becoming.
Under European law don't you have to mark AI generated content as such?
Is that self reported? Does it apply to content created by a U.S. based AI model? Posted on a U.S. website? Read by someone in the U.S.
I am not anticipating people self identifying slop.
How do you think they will verify this?
With AI?
What baffles me, isn't just that people don't see the low quality writing themselves, but that they must also be able to identify other's shit "LLM writings" right? And realize how what they put out sounds exactly the same? Or did people completely stop reading what they've "wrote" before hitting that magical submit button?
People like this generally don't read, for pleasure or edification. Their posts are advertising, designed to work by repetition. They're not necessarily looking to persuade smart people; they're going after the same sort of people who want to fix problems with this one weird trick (doctors hate him!).
It blows my mind. A customer of ours, whom I respect the technical opinion of, drops fully unedited Claude posts on LinkedIn now. I don't get how he thinks they read like anything but slop.
if they had been half decent at writing before they would notice the quality issue, not being able to see it is also a writing skill issue, now it's just kind of spammy as people are just reaching for the tool in ai nighsosis
Yeah I don't quite understand it either. Where's the intersection between people producing AI slop and people that have never read AI slop? I guess it's bigger than we thought... Or maybe they just don't give a shit. That's probably it.
I suspect people who are putting out LLM "writing" are people who weren't trying to communicate in the first place.
They were likely hunting for a better job, chasing the thrill of a million views, or similar - seeking a side effect of writing, rather than seeking to help someone else understand what was in their head.
I worked at LinkedIn (briefly, before retirement) and the product had been useful to me for years before that, so I used it a fair bit but not much anymore since it doesn't serve any of my current needs. There were always awful million-view posts written by humans, useless influencers promoted by the algorithm. That was always garbage, but to an LLM those posts look like success, because engagement, so the LLM tries to replicate that garbage, does a mediocre job, and the result is this sort of weird garbage. Basically the LLM appears to be high on its own supply. Problem is the viral stuff was always what made me want to run out of the room and lock the door, so even if the LLMs did a better job of replicating that garbage, it would still be garbage, just a little less weird.
Part of it also with seeing the feed is that it is basically useless for me. The only interesting thing is when it automatically tells me someone I know got a new job someplace. All the rest is pure and I mean pure slop. No one I actually know ever posts anything to linked in. It would be like making a facebook status update these days. Like who does that, other than the influencer types of course?
When I log into LinkedIn, it seems like a vast majority of the posts are there to farm credibility and build a personal brand.
Pre LLMs, I found it all to be pretty gross. Post LLMs, my reaction to being on the receiving end of that is to find it pretty intellectually insulting.
There are ways in which using an LLM is branding suicide.
So much this. My average linkedin in exprience is a sprinkling of updates from people i personally know, in an oceon of inane "content" by ~$0 net worth people with 40 word self professed titles.
I am still baffled that microsoft is positioning that asset as a professional product. Imagine if bloomberg terminal landing page was a scrolling feed of aspiring influencer bullshit.
> branding suicide.
Or not. I mean, if you had no voice and nothing to say before, you can amplify that signal 10x now. Basically, you’re boosting your brand as a tasteless brainless schlub.
I can always tell when someone uses an LLM to write and never gave the LLM proper context on how to write (i.e., writing samples, skills files specific to writing, etc). These are also the people who tend to complain about AI's writing style.
You can get pretty good results if you take the time to tune it. Tropes and the like still sneak in but if you take the time to edit and rework things, it ends up being a pretty good workflow (especially for stuff that's more procedural, not artistic).
So there are people who actually spend time on the LinkedIn social feed?
Man, it has been purely made of fake content, even since before LLM were a thing.
And the notifications are so senseless that everyone disable them -- and people don't even answer the private messages.
There is nothing social about LI.
> with LinkedIn we know that we’re getting something ... that isn’t totally crooked
Citation desperately needed.
LLM writing is boring. Write it yourself. You are more interesting than a LLM because you make mistakes, do stupid things, don't go for the average content, and can be ridiculous in a way that an LLM can never be. LLMs are great for large tasks which would take you several hours or several days to complete that are routine or tedious. Even then, you have to make it human by putting in your own viewpoint and misguided ideas. I really liked the article. I use LinkedIn a lot, I think it is a better platform than he is describing. There is some dumbing down so it meets the average person. That is its strength and weakness.
You’ve made me realize that LLM’s greatest strength is iteration. It helps you iterate really fast, but it’s not original. It doesn’t help you think of new ideas. So writing about the things you know is great and using AI to learn about things you don’t know and then writing about them is also great.
The way I explain it to young people is that AI is a force multiplier. Everyone gets the benefit of that force multiplier, but it’s that initial force that you need to build up now. Preliminary knowledge of domains, things that the AI doesn’t really understand. AI can help you increase that initial force. So use it to learn new things, not just do things for you.
I was blown away by the fact that AI solved an unsolved math problem, but it made complete sense when it was Terrence Tao, someone who has a PhD in math, that was guiding that agent to that solution, so that initial force is important more than ever.
“…at worst, they will cause people who see it for what it is to question your authenticity entirely.”
love it. I mean, if a world is going to exist that treats LLM-assisted writing with serenity, the content creator should consider themselves mandated to describe how they used AI to produce their content. Do I need the prompts? Not necessarily, although bonus points for transparency if they do share prompts. But just a high-level articulation about how they leveraged AI, so I as the reader don’t have to lose time wondering how much of the content & (like expository & analysis for nonfiction, plot elements for fiction is the author’s own and how much is the LLM’s.
> When you use an LLM to author a post, you may think you are generating plausible writing, but you aren’t
There's some toupee fallacy at play here. The author probably reads a lot of LLM assisted content without batting an eye, but only spots the worse of the LLM outputs. There's a big difference between "write a post about _" vs "improve the grammar/style of my post: _". It's a bit like saying that movie CGI really sucks because you can always tell it's fake.
Why was the title truncated and its casing changed? The original title is
> Your intellectual fly is open
which makes more sense.
Your title made think that a software project was open sourced or something.
The title of this submission has been on a real journey.
I believe HN does some automated editing of titles to avoid clickbaity titles being displayed. May be that.
It does, submitters can edit/revert those changes and they stick
I assume the filter deletes "Your" if its at the start of the title.
I’d rather be embarrassed by my writing than embarrassed by an LLM speaking for me.
Same here.
I remember the first time my email client made an unsolicited suggestion about how to compose a thank you note to my grandmother. After seeing what it had autocompleted I thought, "oh, wow that's a lot better than I could do" for a split second before realizing, "wait, what the hell is wrong with me getting a computer to write a thank you note to my grandmother!"
I've never even tried to use AI to write since. I'd be so embarrassed.
Has any research been done to try and determine the percentage of the population that are able to detect obvious LLM-generated text?
Interesting question for sure, but for now the writing is just bad. So people might not know it's LLM prose, they just notice its poorly written and engage with the ideas less
> obvious
Who decide what is obvious or not?
majority of population dont read anything at all. Most they a read is perhaps labels and signs at a supermarket.
I think there's probably a large chunk who "read" Facebook and skim the linked articles poorly.
People are still on facebook?
Well, sure, but with his fly open the LLM user harvested more wheat. I guess everyone would walk around with fly open if it allowed them to do twice the amount of work or more.
I remember when I saw it the "list with things prefixed with emojis" the first time, and started doing it myself thinking it was a great way to break down things visually. Looking back at the things I wrote then people would definitely assume it's LLM.
I sometimes see posts on LinkedIn that look LLM generated but also makes me think it's someone who has seen a post, thinking it's a great way to convey a point and tries to adapt the style, without realizing it's a smell.
Whenever I see a cringe post on LinkedIn, I medically block that person or mark that post as not interested. That’s the only way you can judge the algorithm not to show those posts
We hired a new guy at work that's leading a specific platform initiative.
In person he's very articulate, able to communicate abstract thoughts clearly, states clear goals and how he's learning about the business to develop a plan.
But dear lord everything he writes in email and slack reads like it's copy paste from chatgpt. It's unnerving.
Had a similar experience when my talented lead engineer, who usually grunts in monosyllables and writes terse responses to my apparently intellectually frivolous questions, started adding flowery emojis to his email.
I'm gonna be contrarian here and say this is a Good Thing. It's very easy to tag to tag this sort of writing, and that makes it an absolutely great signal about hte people who post it. The more vacuous or frequent it is, they less weight you can accord them and the easier it probably is to get past their filters and make use of them for your own ends.
Is that morally bad? Guess that depends on exactly how you make use of them and what those ends are. If you an encourage an idiot to praise your competitor's product in order to get a reverse halo effect, that's kinda bad. If you give them some empty flattery in order to bridge a contact with someone you actually want to connect with, that's probably OK. The point is that uncritical LLM repetition tells you something about the person and lets you see past metrics like the apparent amount of wealth they have, the intimidatingly deep resume, or the degree size or centrality of their network. Two things, in fact: they post any old thing that brings in the clicks, and they're cheap. Previously many of these people probably paid someone to ghost-write their commercial affirmations.
Conclusory zinger goes here - punch up the dramatic contrast PS I'm reducing your fee to 10c/word, hope that's OK. Inflation
Personally I wouldn't be writing an open letter to what is effectively a bunch of spam bots. I don't think they'll read it, or care.
I'm not morally/philosophically opposed to someone generating text with AI. I just don't want to read something that isn't really "owned" by the person creating it (that is, the writer knows what he is sending me and endorses all of its contents), and because I can't discern that from people I don't know, I tend to view it with suspicion by default.
This is perfectly valid position to have. I also feel the same way.
Unless I'm directly interacting with the LLM, someone should endorse the content.
I see, we are trying to re-establish ownership over the em-dash. This brings me joy.
The only place where I use AI for writing is to summarize press articles; this allows me to legally have a mirror of a press article I am commenting about in my blog (whenever I use AI, I make a disclaimer that the prose is AI-generated; EDIT: I also link to the original article, and an archived copy if available). Now that archive.org doesn’t mirror many press articles, and now that archive.today is having a lot of reliability problems (right now its CAPTCHA is down because it has gone beyond Google’s limit for CAPTCHA use), an AI summary of a news article (along with a link to the real article) is the only way to guarantee the reader will get this gist of the article I’m commenting on.
I also use AI to colorize old black and white pictures when discussing historical events in my blog, as well as the occasional AI enhancement of an old grainy and/or blurry photo.
So does Fair Use in the states?
Actually, it’s because, as per Feist Publications, Inc. v. Rural Telephone Service Co., facts cannot be copyrighted.
This legal trick only works for rewriting an article reporting on a factual event—since the events are uncopyrightable facts, the only part of the article which can be copyrighted is the stylistic writing.
Let me quote from a recent legal opinion on AI summaries (The New York Times Company v. Microsoft Corporation et al 2025):
>>> Exhibit 11 to the CIR complaint provides website links to articles that CIR alleges were unlawfully abridged by defendants in their ChatGPT and Copilot outputs. (CIR, FAC Ex. 11.) Examining the similarities between those outputs and the corresponding CIR articles, including the “total concept and feel, theme . . . sequence, pace, and setting,” Williams v. Crichton, 84 F.3d 581, 588 (2d Cir. 1996), the Court concludes that the “abridgments” contained in Exhibit 11 are not substantially similar to CIR’s copyrighted works as a matter of law.
The alleged abridgments are detailed summaries, usually in bullet point form, of the facts contained in CIR’s articles. Those summaries—which differ in style, tone, length, and sentence structure from CIR’s articles—are not “substantially similar” to CIR’s copyrighted works. They present the “facts in a different arrangement”—bullet point lists or short summary paragraphs—“with a different sentence structure and different phrasing.” Nihon, 166 F.3d at 71. In short, the abridgments in Exhibit 11 are not substantially similar, qualitatively or quantitatively, to the original CIR articles as a matter of law. The Court therefore grants OpenAI’s motion to dismiss CIR’s claim of direct infringement under 17 U.S.C. § 501 insofar as it relates to the “abridgments” contained in Exhibit 11.<<<
But I meant, isn't saving articles for personal record a legitimate use of Fair Use? Do you really need to paraphrase them first?
Copyright only covers distribution, but I distribute the AI summaries on my blog. With paywalled articles and the usual archives having serious reliability issues, it allows readers of my blog to get context about articles I comment on.
Here’s an example: https://samboy.github.io/blog/entries/2026-08-28.html
The links are AI summaries, which, in turn, link to the original articles, but, in some cases, the original articles are paywalled. For the ones which aren’t paywalled, having a local summary prevents link rot.
Fair use covers quoting someone to comment on them. For example, in New Era Publications International, ApS v. Henry Holt and Co., it was ruled that quoting Hubbard saying “The trouble with China is, there are too many Chinks here.” was fair use, since the book in question was commenting on Hubbard’s personality, and could only reasonably do so by directly quoting him.
From that decision:
>>> these brief quotations from unpublished copyrighted work display a compelling fair use purpose. [...] These quotations are in mockery, to show Hubbard's bigotry, bias and coarse lack of taste. This is not an instance of the biographer/critic free riding on the creative talent of the subject. <<<
> anyone who has seen even a modicum of LLM-generated content (a rapidly expanding demographic!), the LLM tells are impossible to ignore
Isn't this likely to be a temporary situation though? Are we at peak LLM writing quality?
It’s never been easier to differentiate yourself as a writer. You just write with some personality. People don’t want writing that doesn’t look average, but looks novel.
I despair much less for human writers than a did maybe a year ago.
> You just write with some personality.
I used to do that, then someone on HN basically said I had mental problems because of the way I wrote the article. Best part is Claude told me to tone it down a bit, and I completely ignored its advice. Wouldn't have happened if I had listened to the AI.
I've also been called a schizophrenic on a GNU mailing list because of my idea and the working code I submitted. Caused me to literally quit the list on the spot. Best part is the maintainer eventually implemented his own version of it.
Since then, others have encouraged me to keep it up, but I just don't feel comfortable anymore with this "just be yourself" nonsense.
Yeah, for some of us, that is bad advice. The version I can most safely implement is, "Be your best self."
> just write with some personality
I think that's the first time someone claims that "write with some personality" is some easy thing you just learn somehow. There are authors out there, even ones that make a living on their writing, who still haven't learned to "write with some personality".
What exactly does that mean and how concretely can people actually do this in practice? A few "tips and tricks" might be more helpful than "just write better" or similar stuff.
Some things that help me.
1. Be vulnerable and share your mistakes. Avoid a triumphalist “everything works” PoV
2. Write about your actual lived personal experiences.
3. Try to have a sense of humor
4. Have an informed opinion or PoV - strong opinions held weakly.
5. Be casual. Don’t treat a blog like it’s a research paper.
Thanks a bunch! I actually tried "writing more personal" just some weeks ago, and the feedback I received so far been relatively OK I think, except some people apparently think the part about the drugs to be childish. May I ask what you think of it, if it does sound like it's written with actual personality? https://emsh.cat/en/the-people-are-still-there/
I won't claim to be a professional author or even good, but lately I've been trying to get more into the "it's a person who writes actually" direction and this was an exploration into that, so any sort of feedback would be most welcome, if you have the time!
Nice list. I'd add: Have an audience in mind, even if it's future you, and write directly to them.
Upvoted for optimism. Plus, I think I agree with you most days of the week. Some time must, of course, be set aside for existential despair.
If a human should read it, a human should write it.
I asked the latest AI model and destroyed consequent amount of water what "fly" is in this context.
Even worse in the corporate world. Most emails about some launch announcements are now entirely written by AI. TTFED (Time to first em dash) is usually like 30 words or less.
What once has been a thoughtful email trying to describe in few words why something is launched and how it might help you etc. is now almost a novel with more paragraphs than substance within the tool being launched.
This makes it almost impossible to stand out as well. Where in the past someone could create a grea looking announcement (eye-candy) and think deeply about what to write there, and then hopefully stand out in the sea of mediocre ones, now every little email seems like it's a multi-million $ SaaS being launched. Just last week we launched an internal tool which was in development for months, and literally a handful of people even bothered clicking the links within the announcement.
This is being one-upped still by leaders writing big project plans for 4-5 months ahead, using AI. Everything from the inception of the project(s) is AI. It has bizzare timelines, more codenames than actual people working on it, the vaguest descriptions of what the things will do etc. Then this is trickled down into the teams, and they... to no ones surprise, throw more LLM at it. Now they start working on the LLM project plan using claude etc. The end-effect is baffling in all sorts of ways (quality, ui/ux, all pages looking different), AI generated images and more. And then, finally, they colaborate on big announcement emails using AI. And if you don't share the optimism and try to explain why this is silly, you're an AI sceptic...
True story from within one of the biggest companies in the world.
I personally use an LLM mostly after I have written my posts/thoughts to fix grammar and punctuation.
The people who are going to be successful in the coming economy is the ones who can channel all the knowledge and experience of an llm without shame. If you are smart:
"... in the style of a my writing"
I've been doing that. It helps that I have a substantial human writing corpus to draw from. Results are still mixed. Models simply can't resist adding some LLMisms. Pangram still detects some of it but not others, and also misdetects some of my actual writing as AI generated.
Honestly, I'd rather the shame and hate just went away instead. It's seriously exhausting and I'm starting to feel tempted to just give up and either start using AI more heavily or stop writing altogether.
Another AI-as-amplifier vs AI-as-substitute article.
Wondering what happens when tomorrow AI accepts and learns from the feedback and gets trained on all the failed initiative as well. It wouldn't be so hard if failed (something which wasn't right at all or hasn't got the traction) projects and ideas are all listed somewhere for an LLM to scan through. LLM may finally figure out how to add personal scars, hard decisions and personal insights which can be personalized further.
Also I don't think it might be so undesirable for an org, if there is a system which observes and present hard facts based on last quarter or year JIRA (pi planning and sprint planning) and commit histories.
It’s the best guy around who tells you when your fly is down, thanks.
I did writing courses as elective in college, the biggest lost truth is that there is no one correct writing style, (you build) being a writer at any capacity means cultivating your own writing voice, (yours) which is an expression of who you are. (voice)
Why spend time on linkedin? It's an intellectual wasteland.
I'd rather read reddit than some ghostwritten thoughtleader piece
We seem to be very sensitive to style. It can be rather distracting. This writing contained 7 exclamation marks and 9 colons.
As for em-dashes, there were 7.
LLM use is like Instagram use.
Both are generally most indulged in, by the intellectually and educationally less fortunate.
Your fly is also open when you indiscriminately fling accusations of LLM writing around. I know it isn't that easy to detect because I get falsely accused of it. To some people it seems to mean text written in complete sentences without glaring errors and maybe a few uncommon word choices. It has become an all purpose excuse for declining to engage with the meaning.
One of my neighbors sent a letter to the HOA president, who refused to even acknowledge it, on the grounds that it was written too well and so must have been AI assisted. I don't know if that was true, and don't much care, because I do know it contained valid issues that deserved a response.
> Your fly is also open when you indiscriminately fling accusations of LLM writing around
Not.. really?
I kinda see the point you seem to want to make, but the "no U" opener makes it hard to do that.
Beside that, HOAs - from what I heard of them - will use any reason they can make up to ignore what you want from them, so I'm not sure if that has anything to do with LLMs.
I struggle with this. I think people just will not want to read what LLMs output, plus perhaps you can still analyze things better than an LLM, although that seems skeptical.
My situation is writing about futuristic long-term business strategy ideas; can that be replaced?
Is this unique at all - s-1.site
I have found LLM’s to be a really good thinking partner for writing. I start with a rough idea then proceed to argue for hours challenging ourselves and throwing away a lot of drafts until something materially useful emerges. It is a very different approach to writing… much morel like attaching a debugger to my thought process.
(2025)
Also in the author's latest post[0], they took this[1] self-reported preference poll seriously, which makes it very hard for me to take their articles seriously.
[0] https://bcantrill.dtrace.org/2026/09/05/the-revolt-of-the-re...
[1] https://writethatblog.substack.com/p/dev-reaction-to-ai-blog...
And in that piece, I think I explained why it's important to take it seriously even though (and in fact, because!) it's self-reported?
No, you explained why the selection bias doesn't matter, which I agree. The self-report data still is completely worthless in the first place though.
People are really, really good at lying to themselves, let alone to an online poll. They'll tell you that they prefer imperfect or even bad writing as long as it's not AI slop, just like how they'll tell you they like healthier food, they prioritize personality instead of look for potential dates, how they use LLM "only as a spellchecker", and how they use tiktok for educational videos. As long as there is no stake, people will just say what make they feel better.
Self-reporting data for human behavior is just noise.
I mean, fine, it's anecdotal -- but the numbers are also overwhelming (and don't seem to disagree with the tenor here).
And look: you're obviously free to ignore me because you feel that the survey data is "completely worthless" and just slop your way to success -- all I'm doing is trying to explain why you shouldn't expect me (and people like me) to read what you create.
I don't agree. Good writing and good ideas are entirely different things. Just read all the beautifully written, inaccurate stuff by the NYT or the Guardian.
Also, non-native English speakers have to use LLMs to share their views so that they are not judged on their writing.
This post is also a good example; it's well written, and I enjoyed it. The idea could have been expressed simply as: "People can smell LLMs in your writing, it makes you look disingenous"
> I don't agree. Good writing and good ideas are entirely different things. Just read all the beautifully written, inaccurate stuff by the NYT or the Guardian.
I'd be very curious of some examples when great ideas are being presented in bad writing (or bad speaking for that matter).
> Also, non-native English speakers have to use LLMs to share their views so that they are not judged on their writing.
weird statement. If they use LLMs they will be then judged for both their inability to write in English and for their usage of AI. Not good.
One example that comes to mind is math textbooks – the early textbooks in a field are usually much worse than later ones that come along. To pick on one, I think most people who've read both would agree that the commutative algebra section of Lang's Algebra is much worse than Atiyah & McDonald's Commutative Algebra book, despite covering basically the same ideas, theorems, etc.
I think this is due to the fact that the language game that we have collectively decided to play, with AI, is not yet mature enough. In the language game of “giving a gift”, for instance, there is a distinction between giving a gift that you made yourself, and one that you bought at the store, but it is not that great, nobody cares a lot anymore. Right now, the language game of publishing words on the internet is strongly associated with the notion that these are “my words”, and the fact that they might have been written by a machine instead, introduces some discomfort. But it’s possible to imagine a future where this language game will have evolved enough that the notion of authorship by a particular machine entity, will not in itself be such an important issue, anymore.
Gerald Ford references - pop culture wow.
Personally I never use an LLM or "AI" for any reason. Don't find them useful and, like others, don't find their writing compelling or interesting. I can understand using it as a coding tool, but can't fathom anyone wanting or needing to filter their own thoughts and ideas through it. Is it really that difficult for anyone to send a letter or an email without machine assistance?! If someone sent me a machine generated email I would view it as being no different than any other spam message. In my opinion doing so shows a lack of respect for the recipient whose time is being wasted with machine generated trash.
If you never used any particular tool, you probably wouldn't find it useful, right?
I asked Gemini about my unicycling this morning, and it coached me on weighting the seat; e.g., looking ahead instead of down helps unweight the pedals.
That's true. I've also never used a wheelchair, or a variety of other tools for people who can't walk, because I can walk without them. The same holds true for communication.
This is basically the main argument of On Writing Well, Zinsser, William Amazing book.
Much like our voices, DNA, odor, and personalities are unique, so is our writing style. When I read a wholly-human writing, I get a sense of what kind of a human I'm dealing with. Using LLMs will distort this signal, so readers worth their salt will easily skip the piece, because we don't want to acquire LLM-like writing 'style' (reading affects writing).
As soon as I sense Slop, I'm done, this person refused to think when writing, why should I waste my time reading it then?
I wanted to share my project, but for some reason, the site won't let me... I guess my karma is too low—ha ha ha! We're all like kids trying to get a good grade.
I think one of my first posts ever was a share. My guess is there probably a new account timeout to prevent spam, and you are affected for having made your account today
This post uses far too much punctuation. Makes it jarring to read.
Perhaps the author is trying to distinguish themselves from LLMs.
So many people seem completely blind to LLM writing, they are fine sharing it on social media or at work. I think it’s people who are “not details people” and are used to judging something without reading it based on superficial cues which is what LLMs are optimized for.
And of course anyone who actually reads stuff is disgusted by it, so we quickly end up with a situation where content is piling up and nobody is consuming it which is basically dead internet theory.
On LinkedIn specifically I went from reading it regularly to almost never touching it, previously the nonsense (“the interviewer was the dog”) was still tolerable enough to flip through for updates and I found the platform useful for business leads. Now it’s just a feed of pure slop, when I do open it I just close again after reading a post is two when I remember how bad it is.
I love the analogy. Thanks for sharing!
writing well still works. it also still can be quite hard depending on the circumstance.
Yes, yes, all is true here. But you forget: Half of America is illiterate. "According to data analyzed via the National Literacy Institute, roughly 54% of American adults read below a sixth-grade reading level."
So, those folks aren't checking anything, and certainly wouldn't know the difference, or care.
LLMs are not defacto bad at writing and are not defacto distinguishable from human writers. Lots of people are just lazy, never customize anything, and tell chatgpt to make them a post about x to be a "thought leader". Most are even lazier and blatantly rip off some else for their own means.
However, it is perfectly possible to have an LLM imitate an existing corpus of writing (yours or someone else) and with a good prompt, idea, and editing, to produce high quality writing (in every sense of the word) with an LLM.
If you're going to put that level of effort in, just write the thing yourself. It will be a comparable level of effort and you'll learn something doing it. Your readers will also appreciate hearing your real voice.
Well, no. If I can get an LLM to write like Asimov, that's better than I will ever do, honest.
It's been odd as I've seen colleagues I generally thought of as intelligent and self aware post glaringly AI generated cliche LinkedIn corporate slop, even accounts from generally respectable and innovative tech companies. I don't know whether they think no one can tell or that they don't care if it is obviously AI? It's truly bizarre and now seems to be the vast majority of posts there.
The people posting this content are intellectual eunuchs, so it doesn’t really matter if their fly is open.
Interesting quick read/take. Especially the LinkedIn analogy. Even though I am WAY high on the AI fanboy and low on the LinkedIn side of the bell curve, I can see what the author mean.
Wait, you're complaining about AI-generated content on LinkedIn? The place that was 99.9% organic human content slop even before there were LLMs? It was always all form, no substance.
People, such as the author, who are smart and competent--be it writing, math ,coding etc.--I think systematically underestimate the difficulty of said task for a general population. Things they take for granted, such as writing well, are surprisingly difficult for large proportion of the general population
Just like LLM-generated blurbs are now a cliched trope, so are short blog posts about people experience in a world with a lot of LLM usage. And this is one of those.
No clue why people use linkedin posts serious?
I mean, i check it to see if a recruiter has something interesting but otherwise?
LinkedIn is also suppressive. The algo for maximizing non-confrontational screen time is strongly tuned against: critical posts and comments. Real bringing of information from outside the bubble. Links are severely suppressed.
To me, LLMs struggle with the most important aspect of technical writing - conciseness. As Einstein put it: “If you can't explain it simply, you don't understand it well enough”
Amusingly, there’s a plane rental company called OpenFly.
I wouldn't ever use an LLM for my own writing. But I'd use, after edits, for letters to bureaucrats, landlords and other types I'm forced to deal with.
The thing is, the author may care about Linkedin but the relationship that many people have with linkedin is as a place they have to be. "Hmm, I need a blog to enhance my career but writing, urg. I know just the thing...". Which is to say, it's not strange Linkedin is going to be filled with crap. None of my actual friends on Facebook post crap 'cause there's no incentive.
The HN title ("Intellectual Fly Is Open") made me think it's about some startup starting to sell subscriptions. That title should be de-editorialized. Was it LLM-generated?
Is LinkedIn a normal social network? It seems like an absolute dumpster fire to me (it even has shorts now!)
sometimes I feel like just writing in all lowercase without fully proper punctuation to signal that it's a real human mind
This feels like satire / parody. Praising LinkedIn. The text itself containing multiple LLM tells, especially in the parts complaining most about LLM writing.
Maybe not. Could just be kind of accidentally ironic with the author picking up LLM quirks from using them a lot.
Funny in either case.
the 1900s called, they want their political references back
I guarantee you people dislike my writing way, way more than they would if it was AI generated. People need to look in the mirror before telling others not to utilize LLM's for writing.
ai slop is sh8t. but using ai to write thoughts in readable way, with flow, no issues on my side.
my english writing is quite bad. if i write article i myself cannot read it. ai helps with the flow, review editing.
nowadays i record audio, stt and then let ai flow it properly.
i hate blog.md like this, given a topic and it does everything.
it's all about money, they write some attractive stuff, they get some views, likes follows, maybe someone will say "this dude is smart". but most times you get idiots liking the stuff, and likely detrimental because a smart person would see is slop, and they also know you don't know people know is slop.
I was fired from my job at spotify for taking fmla when i my dad was dying from cancer. I've been marking the my directors posts on linkedin as 'looks like ai slop'.
Talk to a lawyer about that firing. You might have a case.
i did. They said i had no case since i wasnt fired during fmla and there was no proof that it was the cause.
You are right in every sense, but it seems you used llm to write this post, lol!
What makes you say that? It reads naturally to me and Pangram flags it as 100% human. I've never heard an AI use the "fly is open" analogy.
That's a suspicious number of em-dashes for something that's allegedly against LLMs.
I was thinking that, but the use of em-dashes is different from the normal LLM use.
If this is straight out of an LLM I am impressed
You think it's the em dashes and not the absurd Gerald Ford and Richard Nixon analogy?
Addressed in the article...
LLMs write better than most journalists. It also writes better than majority people in the world when it comes to English as majority are really bad in writing.
And of course, LLMs write as you prompt.
Personally, I'm happy to read broken English from non English people or good English from English people. But I'd rather read LLM writing than read typical long form perfect English journalist crap.
> LLMs write better than most journalists.
I used to think this, but now that I've been trying to use them to help with writing, it turns out they are not necessarily that great on the deeper level. The grammar and the surface style are impeccable, but they often struggle with continuity and carrying on a point or an argument.
I've come to trust their grammar corrections implicitly. So much that I've setup a "Fix Grammar" command with Obsidian Copilot that I'll just let it run on a whole chapter (YOLO mode):
About a year ago, all three big AI vendors could end up with strange "corrections" that would throw you off, but not anymore. As far as I'm concerned, they've solved "the grammar checking problem."The complexity in this command is there for phi4:14b and qwen3:14b which I run locally via ollama. If the text doesn't have any Obsidian callouts or similar, fancy stuff I just use either of those and they do a fantastic job in seconds. For Big AI (e.g. Gemini, GPT-whatevs, Claude) you can literally just tell it, "fix the grammar." No need for the lengthy command.
NOTE: I am decent with English grammar so most of what needs fixing is typos I didn't spot or misplaced commas and periods inside/outside of quotes (I always screw that up without thinking—even though I know the rules! LOL). Occasionally, Big AI (Gemini, specifically) have disagreements about whether a comma is necessary in a particular spot but it's always of no real consequence.
> About a year ago, all three big AI vendors could end up with strange "corrections" that would throw you off, but not anymore.
Without a history of diffs created by this method, I don't believe you.
> I am decent with English grammar so most of what needs fixing is typos I didn't spot or misplaced commas and periods inside/outside of quotes (I always screw that up without thinking—even though I know the rules! LOL).
The rules there vary by style guide and are not objective.
LLMs seem like massive overkill for something the red and green squiggles can already accomplish.
Good writers break grammar rules when it helps convey a point, emphasize an idea, or just makes the writing more interesting. You are completely missing out on that.
With current LLMs, the continuity/carrying/context management is _your_ task. When AGI is achieved etc...
> But I'd rather read LLM writing than read typical long form perfect English journalist crap.
The problem is precisely that LLM writing is that exact thing (at least by default), but even more so. Longer-form, more "perfect" in some technical sense, and yet crappier.
not all of them.
and you can always just ask to be concise.
> LLMs write better than most journalists
In my personal experience I strongly disagree. Curious what publications you read wherein this is the case.
I suspect the commenter prefers the concise, factual summary of an LLM.
It is true that modern journalism on the web is trying to get ad impressions. After all, this is how we ended up with clickbait. Sure you still have the Economist and Atlantic (which aren't beyond criticism), but your local county paper is an absolute mess. The content is stretched, meandering, and designed to keep you scrolling through more impressions.
> concise, factual summary of an LLM.
Pardon; the what?
I would almost never transmit LLM output without a decent amount of editing. But I also don't understand people who paint it as this horrible offensive unreadable slop.
Calling it better than most journalists is extreme (and probably the reason for down votes), but it must certainly be better than the average person.
We're hearing this criticism from the highly educated professional class people who bother to have their own blog or otherwise spend their time talking about technology online. I mean come on.
To most people, the LLM must feel incredibly empowering, like us wearing a mecha suit. Would we always show restraint and only apply our newfound enhanced physical strength in carefully considered situations?
> But I also don't understand people who paint it as this horrible offensive unreadable slop.
When a piece of writing is full of LLM tells, I find it as offensive as any formulaic writing, except that the LLM style has quickly become pervasive, much more so than any other type of formulaic writing.
I’m not reacting out of some general dislike of LLMs - I use them daily. I’m reacting because I don’t like terrible writing.
Well said. When I realize I've been reading LLM filler and leave the page it's the same reaction I would have pre-2021 thinking I found a good source for something and realizing I'm actually reading blogspam.
The difference being this variant of blogspam is produced with basically zero cost and thus the infection has spread far beyond SEO into every UI surface with textbox. Plus there are now passionate defenders who insist finding their blogspam unpleasant to read is disrespectful or anti-progress somehow.
They are worse than most people at communicating ideas.
Please read past the LLM-ness of any given post.
The message of the post is what _should_ hopefully matter to a reader.
And, not the fact that it was written by an LLM.
I am a technical guy at heart; also an introverted extrovert. I hate writing docs that are to be written to satisfy someone else's metrics. For it to be a tickbox'ed item.
I delegate that to an LLM. I want to spend my time solving interesting challenges instead.
So, Mr. Cantrill, you got a problem with that? So be it.
> Please read past the LLM-ness of any given post.
I just can't. LLM assisted posts are often so long, and the hints are usually obvious right at the start.
They're so unpleasant to read. "It's not X, it's Y" etc are bad because the comparisons rarely add anything at all to the message. It's filler that wastes my time. At that point I'd rather see your original prompt.
Why should I invest several minutes of my day reading something if I've already seen evidence that the author doesn't respect my time?
It's not that an LLM wrote it so much as the author couldn't be bothered to clean it up and remove the garbage cliches before publishing it.
Some people actually want to enjoy their moment to moment existence on this planet and not read garbage someone shat out.
It's just digital pollution.
I hear ya. However, that ship has sailed a long time ago and succumbing to this very idea is exactly why puritans in any form will not survive the next 50 years.
I don't think my survival hinges on my reluctance to read AI slop.
The ship has not sailed, I can just find a human who wrote words, and read them. And if they aren't human words, I will just, not read them.
> Please read past the LLM-ness of any given post. The message of the post is what _should_ hopefully matter to a reader. And, not the fact that it was written by an LLM.
No. I am interested in what a human has to say, not a clanker. If you don't wish to write that is fine, but don't hand it off to the slop machine and present it as though you did anything of value.