I have largely written Reddit off and no longer visit it
after an experiment I did where I had an agent karma farm for me and do some covert advertising. As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer. Many many people (or other bots) had full on conversations with it and it scared me a bit.
I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.
Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
For a while there were a lot of posts from people experimenting with ChatGPT to write anger bait posts on Reddit where they would later edit the post to say it was fake, written by ChatGPT.
I assume they thought they'd be teaching people a lesson by making them feel foolish for responding to AI stories, most of which were too fake to be believable.
However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake. In advice subreddits, commenters continue to give advice on the situation. Some comments would say they saw the notice that it was fake but continue arguing about it anyway.
This makes a feature of Reddit very clear: The truthiness of a post doesn't matter. The active commenter base on popular subreddits just wants something to discuss and, usually, be angry about.
In retrospect it's obvious given that misinfo posts were the easiest way to karma farm for years even before AI.
We do precisely the same thing here. Here's a relatively recent post that, to me, seems obviously LLM-written. It just rattles off some management platitudes:
Yeah, the trick is to do your own curation and go from there.
If you like some authors or journalists or bloggers, go see who they read (trust me they all say who they follow in their own niches) and build from there. You can develop quite a good RSS feed following this method in like an hours tops.
The decline of Facebook is sad. I liked it early on. I used it primarily to follow family and casual friends from high school. When they posted, it would show up on my feed, I read all the posts, and that was that.
After awhile I had to wade through all sorts of nonsense to get to the posts I actually wanted to see, and even later Facebook stopped putting posts from people I follow in my feed. It was 100% garbage. I can't imagine why anyone uses Facebook for anything other than the marketplace.
Facebook is fine if you join groups based on your interests (hobbies etc) and then aggressively unfollow/block anything you don't want to see. It's not really conducive to discussions like Reddit, though. Mostly drive-by comments.
I often hear that about Facebook, but at least it has a "feeds" button that you can press to get the sources you actually subscribe to. The default "home" feed is useless.
I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust. Or rather turn them into little better than comment sections on news sites; thriving but worthless.
I'm active in a number of online communities that are doing just fine but the difference is those all involve ongoing relationships, built over time and with engagement across multiple platforms. I've no doubt this clock is ticking too but it's still harder to fake a user across a mix of text chat, voice and video calls, playing an online game, etc and when much of the web of relationships extends back into real life activity.
But I agree the golden age of easy anonymous connections online has ended.
Note that "attestation through a web of trust" means something like needing an invite from an existing user. It doesn't have to mean mass surveillance.
Private torrent trackers have been doing this for a while. If some number of your downstreams act like shitheads - you get nipped and so do your other downstreams.
This seems like the best way to handle it. Also, smaller communities. It's cool to do the global thing, but once you have 10k active users you can't moderate it with a team of 5 volunteers.
I think the attestation approach works best if there are different reasons for the punishment. Eg someone inviting a turd doesn't ban the person who invited them. Someone going full ai spam should.
what.cd was better. You either got an invite where if you tanked your reputation you'd get banned and risk the inviter getting banned too; or you had to take an interview where you got quizzed on how to properly rip music in a variety of methods and how to ascertain between different qualities of rips (like mp3 bitrates to flac cue files).
If you weren't a bellend on what.cd you got access to certain forums where there were even more and better private trackers. Once you built that trust there were social privileges, but if you abuse that trust you got rightfully banned.
This takes it a step further than what you describe. They keep track of who you’ve invited, who they’ve invited and so on and if there’s enough bad leaves on the tree they just cull the entire tree. It’s a somewhat common practice with private trackers
> Note that "attestation through a web of trust" means something like needing an invite from an existing user.
It's probably better to call this something like vouching and leave "attestation" as the contemptible power grab by megacorps delenda est. The advantage in using the same word for a useful thing as a completely unrelated vile thing only goes to the villain.
PGP’s web of trust was kinda bad privacy-wise in some regards, as it basically revealed your IRL social network.
If my PGP public key has 6 signatures and they’re all members of the East Manitoba Arch Linux User Group, you can probably work out pretty easily which Michael T I am.
Are there successful newer designs, which avoid this problem?
The IRL social network is actually the important part of the trust structure.
The only one of these I've seen that really worked was the Debian developer version: you had to meet another Debian developer IRL, prove your identity, and only then could you get the key signed and join the club.
> The IRL social network is actually the important part of the trust structure.
For Debian-style applications that are 100% about openness and 0% about secrecy, sure.
But if you want to secure communications between pro-democracy activists in China, or you're a Snowden-like whistleblower wanting to securely communicate with journalists - y'all probably don't want to be vouching for one another's keys.
>Then how can you have a community that is welcoming to people who are not part of the ingroup?
Being welcoming to every random person is by definition not a community, it's a free-for-all mess.
A community means communal interests and values, it's in the name. And to guard those you can't just be accepting everyone without vetoing them. That's how it turns to a shit of spammers and trolls and people who want to hijack it and don't share the original cause/spirit. Has happened to forum after forum...
You'd have to be brutal about culling, uninviting and removing anyone who doesn't look like a good fit.
Or have a two-stage process: run very public, very open events that anyone can sign up to an attend. And then invite specific people that you meet at those events that look like a good fit for your community to your private, community-only event.
Which is, funnily (?) enough, how a lot of IRL organizations used to be. And basically don't be of the wrong ethnicity or religion.
It still happens more informally today, of course, but it used to be a pretty (if un-spoken) part of how a lot of WASPy organizations operated to a greater or lesser degree.
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
This seems self evident to me too.
It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
I'd be interested in working on a problem like that.
I have a strong preference for remaining anonymous or at least making it a reasonably high bar to tying my online identity to my personal identity
I would love to be involved in helping to design a sort of "human verified" badge that doesn't necessarily make it possible or at least not easy for everyone to find your real identity
I've been thinking about it a bunch and it seems like a really interesting problem. Difficult though.
I suspect there is too much political and corporate will that wants to force everyone online to use their real identity in the open, though
I'm not sure that it would be too hard technically... basically, auth+social-network. Basically Facebook auth without the rest of facebook, adding attestation.
IE: you use this network as your auth provider, you get the user's real name, handle, network id as well as the id's (only id's not extra info) of first-third level connections.
The user is incentivized to connect (only) people that they know in person, and this forms a layer of trust. Downstream reports can break a branch or have network effect upstream. By connecting an account to another account, you attest that "this is a real person, that I have met in real life." Using a bot for anything associate with the account is forbidden, with exception to explicit API access to downstream services defined by those services.
I think it could work, but you'd have to charge a modest, but not overbearing fee to use the auth provider... say $100/site/year for an app to use this for user authentication.
I don't think the main challenge is building this system, the main challenge is getting enough people using it to make it worthwhile.
Personally I think it should be a government provided service, not something with a sign up fee. There's actually no point at all in building this if people have to pay to use it, because they won't
> Will they interoperate with foreign governments?
Ideally, yes
But you're right, this isn't likely to happen in real life and I'm just being wishful. Instead we're going to get the much shittier capitalist version of this where every company and government spies on us and we have no expectation of privacy online at all
I agree its a very, very interesting problem. Maybe one of the biggest problems of the coming decade.
I suspect it will be a long process: first there will be goverments that force people to use ID, but that will be abused, hacked and considerably restrict freedom of speech, so after that phase people will start to create better ids.
The problem is really pretty simple: You need an authoratitive source to say "This person is real" - and a way for that source to actually verify you're a person - but that source can be corrupted and hacked. Some people will say "Crypto!" but money != people, so I don't see how that works. Perhaps the creation of some neutral non-goverment-non-profit entity is the way, but I can see lots of problems there too, and it will probably cost money to verify someone is real - where does that come from?
*You need an authoratitive source to say "This person is real"*
Does that even accomplish much? It may cut down on mass fake account creation. But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.
Yeah, that's a problem, you're right. There are some ways to migitate it, but they introduce their own issues. Like say you give someone only 1 ID for their lifetime, they start to spam AI crap, you ban their ID - sounds ok except who is available to police all 8 billion IDs and determine if they're spamming? Who polices the police? What if these IDs become critical for conducting commerce and banning someone is massively detrimental to their finances? Etc. These problems aren't necessarily unsolvable though - but they are super difficult.
If there's only 1 or just a handful of verifiers, then a human can at most go through a few of those credentials before they run out. The risk is of course getting someone else's credential but that isn't as big an issue, especially for smaller online communities.
I just don't see a world where a small community ends up having to deal with a dedicated set of potentially spoofed identities. There are already tools like slow-downs and post limits for new members that can protect against this. HN is the biggest community I'm in by an order of magnitude and it's the only community I know that can't just use a slow mode type mechanic to halt this kind of attack.
Have you considered sock puppets? It's not out of the question to handle with human mods but detecting them automatically is pretty bad if someone is supplying credentials to each one, and sometimes it does take months or years to notice that new user Y is banned user X.
I think sockpuppets are only useful in a community with non-text signals like upvotes and downvotes or likes. These kinds of signals are not necessary and often plain corrosive to small communities. In a larger community they're a great feedback mechanism, but large communities are fundamentally different spaces than small ones and need a fundamentally different moderation approach IMO.
I've seen them used to dogpile in arguments (harder to do since you need to keep writing styles distinct), game votes in forum games or quests, etc. And of course you don't need to use multiple at once if you just switch to a sock puppet every time you're suspended or banned.
> But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.
They can, but ideally they wouldn't be able to make infinite accounts with that authenticated status. So it would still reduce the number of bot posters on the web
There is actually a different problem with this: Suppose there is a major vulnerability in some popular device. 50 million people get compromised; the attacker can now impersonate any of them at will. They go around and create 50 million accounts on various services, or take over the user's existing account on that service.
What are you going to do with their identities at that point? These are real people. If you ban them, you're banning the innocent victim rather than the attacker who still has 49,999,999 more accounts. But if you let them recover their accounts or create new ones, well, the attacker is going to do that too, with all 50 million accounts, as many times as they can. You don't know if this is the attacker coming back for the tenth time to create another spam account or if it's the real victim trying to reclaim their stolen identity.
So are you going to retaliate against the innocent victims by banning them permanently, or are you going to let the attackers keep recycling the same identities because a lot of people can go years without realizing their device is compromised and being used to create accounts on services they don't use?
Yeah that's a big problem. Pretty sure you can see it in real life where lots of old dead accounts with weak passwords on facebook or twitter eventually get hacked. It must be pretty weird to see your dead grampa suddenly start trying to get people to buy some weird scammy crypto.
I guess you could have an eyeball scanner at your computer that only sends out a binary "yes this person is human" to the system every time the log in. That sounds expensive and hackable and just janky though.
Maybe it would result in people taking Internet security seriously and holding companies accountable for data breaches if there were this sort of consequences for it
Crypto could be a part of it. Like you need to sign with an adress that has held some non-trivial amount for some minimum amount of time. As a component of such a system it could cut down on mass or low-effort impersonation.
Money is great at thwarting spam/Sybil attacks. You don't have to raise the price very much to make them fail.
Honestly I think "this person is real" is the wrong goal. You'll never accomplish it without a centralized state or some biometric monstrosity like that thing Sam Altman created.
Yeah, I think "pay to enter" or maybe "pay to be able to post" is ultimately going to be the solution. Then we'll have the paid "gated" social networks, filled with mostly humans, and the free ones will all be bot-swarmed wastelands.
Verifiable credentials are all about this. You need some sort of credentialing body that generates the credential for you, but after that you'll just have an opaque identifier. Any caller that wants to verify whether you're human submits the id to a verifier and the verifier says yes or no. You can also do attestations like age, so gate a forum on 16+ or something. You never end up having to actually give away your name or any other details.
What happens when someone agrees to sell or give away their id? The credentialing body could catch the very worst abusers who seem to be signing in to various sites and services multiple times an hour, but would fail to catch anything else.
I don't think you'll ever be fully free of spam, so you'll still need to filter bad content. If credentials get sold and used to spam, they'll get banned.
How do you ban credentials if they're anonymous? Notice that if you can tell two requests are from the same person then you can do it across services by both of them pretending to be the same service.
Also, what happens to someone whose credentials are compromised? Are you going to ban the credentials of the victim rather than the perpetrator?
world.org is doing exactly that including the privacy aspect.
the iris scan aspect is scary but the alternatives don't seem to solve the problem either.
> It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
The problem here is that the premise is the error. "Prove your ID" is the thing to be prevented. It's the privacy invasion. What people actually want are a disjoint set of only marginally related things:
1) They want a way to rate limit something. IDs do this poorly anyway; everyone has one so anyone so criminal organizations with a botnet just compromise the IDs of innocent people -- and then the innocent are the ones who get banned. The best way to do this one would be to have an anonymous way for ordinary people to pay a nominal fee. A $5 one-time fee to create an account is nothing to most ordinary people but a major expense to spammers who have 10,000 of their accounts banned every day. The ugly hack for not having this is proof of work, which kinda sorta works but not as well, and then you're back to botnets being useful because $50,000/day in losses is cash money to the attacker that in turn funds the service's anti-spam team, but burning up some compromised victim's electricity is at best the opportunity cost of not mining cryptocurrency or similar, which isn't nearly as much. It would be great to solve this one (properly anonymous easy to use small payments) but the state of the law is a significant impediment so you either need to get some reform through there or come up with a creative way to do it under the existing rules.
2) You want to know if someone is e.g. over 18. This is the one where people keep pointing back to government IDs, but you only need one piece of information for this. You don't need their name, their picture, you don't even need their exact birthdate. Since people get older over time rather than younger, all you need to know is whether they've ever been over 18, since in that case they always will be. Which means you can just issue an "over 18" digital signature -- the same signature, so it's provably impossible to tie it to a specific person -- and give a copy to anyone who is over 18. Maybe you change the signature e.g. once a day and unconditionally (whether they require it that day or not) email all the adults a new copy, but again they all get the same indistinguishable current signature. Then there are no timing attacks because the new signature comes to everyone as an unconditional push and is waiting for them in their inbox rather than something where the request coincides with the time you want to use it for something, but kids only have it if an adult is giving it to them every day. The latter is true for basically any age verification system -- if an adult with an ID wants to lend it to you then you can get in.
3) You want to know if the person accessing some account is the same person who created it or is otherwise authorized to use it. This is the traditional use of IDs, e.g. you go to the bank and want to withdraw some cash so you need a bank card or government ID to prove you're the account holder. But this is the problem which is already long-solved on the internet. The user has a username and password, TOTP, etc. and then the service can tell if they're authorized to use the account. It's why you don't need government ID on the internet -- user accounts do the thing it used to do only they don't force you to tie all your accounts together against a single name, which is a feature. The only people who want to prevent this are the surveillance apparatchiks who are trying to take that feature away.
In order to make this viable, wouldn't you have to verify identity repeatedly? What's to stop me from providing a valid identity and then handing my account over to an agent after I'm verified?
That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots. In theory at least. It's certainly more complicated than only that in practice.
If the web of trust only extends to the people who I actually know to be real, then that works -- but it's a very small web.
And by small, I mean: This whole trusted group could fit into one quiet discord channel. This doesn't seem to be big enough to be useful.
However,if it extends beyond that, then things get dicier: Suppose Bill trusts me, as well as those that I myself trust. Bill does this in order to make his web-of-trust something big enough to be useful.
Now, suppose I start trusting bots -- maybe incidentally, or maybe maliciously. However I do that, this means that Bill now has bots in his web of trust as well.
And remember: The whole premise here is that bots can be indistinguishable from people, so Bill has no idea that this has happened and that I have infected his web with bots.
---
It all seems kind of self-defeating, to me. The web is either too small to be useful, or it includes bots.
Critically, it doesn't have to be binary trusted/untrusted, and it doesn't have to be statically determined. If Bill vouched for you yesterday and today you are trusting a bunch of discovered bots, that would down weight the amount of trust the network has in Bill a lot more than if he vouched for you did months ago.
The question is whether we can arrive at a set of rules and heuristics and applications of the system that sufficiently incentivizes being a trustworthy member of the network.
I guess it would have to be something like a service which confirms whether a person already has an account on the site but doesn’t have to track which particular account it is.
I’m not sure if that would work for account deletions though.
There's some work on using phone accelerometer data as a "proof of human," e.g. "move your phone in a figure eight," which I guess machines can't quite do in a human enough way yet.
> without either proof of identity or attestation through a web of trust.
Let's put aside the idea whether it will be the end of all privacy as we know it (I'm not sure if I personally think it's a good idea), but isn't Sam Altman's World eye ID thing supposed to do that? (https://world.org).
How does it work (like OpenId)? Do I have an orb on my desk, or some sort of phone app? I still want to use my desktop to login to HN.
Would it stop this sort of "get human id", past it into .env, so agents can use it?
this eye thing will never work. people in general are realizing the last people we should trust with our personal stuff are tech bro billionaires. they’ve broken trust too many times.
even worse many of them are just plain vocal about their disdain for people in general.
at least from what i’m seeing, people are starting to walk away from online at an increasing rate so i definitely don’t see widespread adoption of his creepy eye thing.
“If McDonald’s offered three free Big Macs for a DNA sample, there would be lines around the block.” - Bruce
I have no idea about the eye thing taking off. But I think your comment is very HN and a bit out-of-touch with regular people. What "you're seeing" is a bubble and not representative of the general population. The eye thing is a slow frog boil and it will be commonplace before you can blink.
Personally I think we need to start utilising the safety features built into AI, to ensure that who we're talking to is a human. We'll start to have to only reply to people who talk in nsfw cursewords (like cocks), or profess their love of capybaras
Identity politics have nothing to do with your actual identification documents. Think: Black Americans being treated as a homogeneous voting bloc, or that all Hispanic voters would be pro-immigration, or "the Evangelical vote".
>I think it's going to effectively kill public chat communities without either proof of identity
How? I have an identity. A state driver's license, birth certificate, social security number. I've even considered getting a federal license before, never bit the bullet. If I wanted to run a bot, what stops me from giving it my identity? How do I prove I'm really me (a "me" exists, that's provable), and not something I'm letting pretend to be me? You can't even demand that I do that, because it's essentially impossible.
Is there even some totalitarian scheme that, if brutal and homicidal enough, could manage to prevent this from happening (even partially)?
I'm limited to a single identity only as a resource constraint. Others more wealthy than I (corporations or ad hoc criminal enterprises) could harvest thousands of real identities and use those. Consensually, through identity theft. The only thing slowing it down at the moment are quickly eroding social norms (and, as you point out, maybe they're not doing that and it's not even slow at the moment).
Digital totalitarianism would prevent it. The moment you were found to be running a bot, your identity would be blacklisted across the entire internet.
You claim this, but you've not presented any evidence. Who would be the enforcement agency for that? Where and how would you train them? Can the money be scrounged up to do it properly? As you blacklist people from the internet, you lose their tax revenue (they're locked out of the economy), but you also make it impossible for them to tell people how bad it was... most of the deterrent effect is gone. But the incentives are only ever growing higher, as people surmise that running their own little bot farm is a way to get ahead when hustling. Any you do hunt down and disconnect are now highly radicalized and desperate, but you've just turned off the feeb's ability to monitor them and intervene.
China gets away with this shit because they've been conditioning their population for 60 years... everyone's eased into it. Elsewhere, not even slightly so.
The web could become a way to indicate identity if public institutions publish for example www.university-country/professors/John. And that implies that John is a professor. I designed a 6000 lines protocol, but anyone could construct that web using hmac(salt+ url).
This is actually my hope for AI-gen content as well. That after it gets so 'good' that people genuinely can't distinguish it from reality anymore that they'll retreat (or return triumphantly rather) to the physical world to gather truthful fulfilling experiences and dopamine.
The fact that reddit enabled hiding your posts is crazy to me. In a time where knowing who's engaging in a community is more important than ever (am I talking to a bot or a troll?) reddit removes even more options to validate.
I interpreted that as an attempt to mask the number of bots on the site so as to not scare paying advertisers into thinking their ads won't be seen by real humans.
They also now hide the number of subscribers. Before you could see if a subreddit was popular or not. Now you really don't know. I think reddit does this so they can promote stuff to the front page for clicks even if it isn't popular.
The problem is that it has become very popular to ban people from a sub based on what other subs they post to. It was turning Reddit into a two-party universe.
The better fix would be to make the support for multiple accounts in the reddit app not so incredibly-shitty, where you're basically logging out and logging back in. Instead, just tell it "posts to this sub use this account, posts to that sub use that account", etc.
There's also a third category where the sub looks organic because the moderator deletes and bans anyone who doesn't post exactly what the moderator wants.
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
Would be super fascinating to watch play out. I grew up before the internet so, historically, I know how to seek out external communities, but by early high school I was deeply entrenched in online life - so I'm very rusty with finding new IRL clubs, cliques, etc. Fortunately my life is full of many friends and I go out frequently, regardless. For those younger people that never had life without the internet, I wish them luck on their search but at the same time I'm very curious to witness their journey.
I'd argue that Reddit leadership, which insulted, hobbled, and wrote off its mods and power users (destroying projects like /r/BotDefense) while doing little to crack down on the proliferation of bot repost content, had a major role in encouraging this. They might even like it better this way -- lots of extra fake engagement boosting traffic stats without messy human drama, which they can then ironically sell back to AI labs as training data.
Let's never forget the summer of 2023 when Reddit forceably removed mods from many major communities and replaced them with corporate shills. That was a major loss of dedicated people who cared more for their communities than Spez's pocket book.
You say that but many specialty subreddits never returned to their pre-protest engagements. Quality has definitely taken a nose dive in these subreddits as those people moved to other platforms like youtube, tiktok, patreon, or just posting on their own sites.
Mods were rightfully upset because they were losing control of their communities when reddit preferred only caring about their upcoming IPO.
I honestly don't think you could remake reddit if you did everything exactly the same starting in 2016. Corporate social media has definitely ruined the individual aspect of social media that is unlikely to return.
No one wants to share on a place with a bunch spammers.
The API changes were put in place for the purpose of breaking, and did break, slmost all external moderation tool software which changed the task of moderating a forum with hundreds of thousands, or millions of users from an impossible Sisyphean task to something that was actually manageable by a dozen or so mods.
The protest came after that so the timeline is not quite correct.
The internet is rather trending in that direction, isn't it? Youtube got rid of downvotes and apparently upload dates, which seems like an easier way to trick people into ads. And Reddit, like you said
If these platforms had to listen to "their customers" (here comes the inevitable comment about how users aren't customers; yes, I know)? They'd all be fired. They'd have to find a new job. They all act in incredibly insulting ways with a too big to fail attitude
That sounds pretty great. If we could just flip 1 switch to accomplish things in absolutes, then that'd be awesome.
I'd like to flip the switches that absolutely end poverty globally, absolutely eliminate guns from the US, and absolutely remove bots from Reddit,
If you can show me where these switches are located, I'll cheerfully go flip them and accept full responsibility for the results.
(Over here where things don't work in absolutes: Some of those bots that got killed were countermeasures to help keep the bad, well-funded bots at bay.)
"Congratulations, sir! Your directive to eliminate all guns has been a roaring success! We've had 100% compliance amongst good, law-abiding people! All of the remaining guns are owned by outlaws! Violent crime has tripled, exactly as predicted! Everything is going according to plan!"
IRL communities have to have some guides because a lot of people forgot how to gather. It can be seen among kids - try to give them soccer ball and see what they do with it :)
Yesterday I was watching people on the street and on the tram. Every other person was staring at their phone and scrolling through something.
That might scare me more than the fact that someone is chatting with an LLM bot online.
(I am pro-ai, use it every day for coding that I couldn’t achieve pre-2022 as I am lame coder.)
Serious question: If there are so many LLMs on online forums, who is doing it? Is it just 1000s of research students or something more nefarious? Is it AI businesses building up evidence that their output is as highly scored as humans therefore "buy our software"?
We're in the middle of an active cold war where countries are trying to manipulate the citizens of rival countries to destroy their civilization without having to fire a single bullet. Anonymous, over the internet mass manipulation, all for some minimal electricity cost.
If Russia is willing to spend cash like that, then of course they're willing to run massive bot farms to pollute any forums they can. I'd be shocked if the US was not doing the same in any way they can. You have to ask why Trump killed Radio Free America as well when it was clearly not an big expense.
Not sure how this relates to the subject in a direct way. Radio Free America was a outlet explicitly created and utilized to spread US propaganda, but kinda sorta barely disguised as a journalistic enterprise (not really, if you were listening to RFA you knew what you were listening to.) Shutting it down seems to be a counterpoint to all of the covert participation of US intelligence on the web which has done nothing but escalate.
It was a head scratching decision that few believe was for the stated reason. Other countries are ramping up their propaganda arms while Trump shut down part of the US'. The reasoning was cost, but that doesn't make a lot of sense in the grand scheme of things. Foil hat types would easily believe it was the puppet doing the bidding of the one that pulls the strings. RFA has been a thorn in despots' side for a long time.
> You have to ask why Trump killed Radio Free America as well when it was clearly not an big expense.
The obvious answer to that question is "because he's a Russian asset". But that doesn't mean the obvious answer is also the correct one.
IMHO, we're seeing another and much more concerning trend at play here... the utter and complete rejection of anything but violence by the far-right. Diplomacy? Development aid? Cultural exchange? All sorts of soft power have been under attack for decades now, and not just by the far-right but (especially when it comes to development aid) also by mainstream centrist parties across the Western world. And it's always pseudo-masculine / "strongman" BS backing the sentiment - Bernd Höcke, German AfD mastermind, comes to my mind with "we have to rediscover our masculinity" [1], so do Hungary's Viktor Orban and his denouncement of LGBT or Trump's entire Œuvre.
I'm not saying that violence or at least being prepared, ready and willing to use it is automatically bad. Far from it. But all the various forms of "soft power"? They have a lot of value, value that the far-right is all too willing to just burn for entertainment.
wouldnt it be more productive to talk about the systemic framework leading to this inflamed state of affairs, and ways that we can tackle the issue on the ground level? perhaps inhabitants of the west would prefer pseudo masculinity to another few decades of migrant influx without corresponding upgrades to social infrastructure. this sort of internal struggle would provide a ripe substrate for foreign agents to perform subterfuge, especially in a screen based world where the narrative can be remotely influenced. conclusively, the population has been convinced that voting far right is the correct decision in their favor, but the question remains, who is it really in favor of? call me a centrist all you like but members of my family were executed under communist regimes so i find it pointless focusing on one side of yin/yang here (in other words, extremists are violent regardless as to their political persuasions).
It's very common for folks to search Reddit to find reviews of products etc. these days. If you can have a bot account post a fake review of how awesome your product us, and have that upvoted, it can pay huge dividends.
In the case of Reddit and HN a lot of it is done by businesses either blatantly advertising themselves or building up the karma they need to effectively do so. I recall reading obviously AI generated replies to news articles written by accounts associated with businesses related to the events in the news. This isn't new in the LLM era. Hobby subreddits are well known to be always full of businesses selling hobby gears and items doing self promotion. It's just that now it is a lot more obvious because of the AI text smell.
That, and probably political astroturfing. Before every election my local subreddit sees a surge of crime stories. Go figure.
I think some of it is account farming, but some is just people buying wholesale into the idea that if you're not using AI for everything, you're gonna be left behind. On the Kagi Small Web list, there's plenty of hobby blogs that used to be normal pre-2023 and are now obviously LLM-written and AI-illustrated. There's also plenty of people on LinkedIn who post AI slop because they think it helps them build a "professional brand". I even have some distant friends who are using AI for responding to friend & family posts on Facebook just because it makes you seem... smart? engaged? I don't know.
It's actively encouraged by some of the platforms too. In Gmail and Google Docs, you have incessant AI prompts along the lines of "help me write this". I think LinkedIn does the same.
People like the above poster who are "just running an experiment" or "trying something for fun" who then wonder why online communities are full of AI now.
Lots of marketing. Not even AI business, just regular consumer crap. They realized that blatantly spamming their product looks bad, so they orchestrate multiple accounts to look more organic. And people actually engage with it.
There are many reasons for influence campaigns, that isn't new. Influencing the public is incredible valuable; that's why so many invest so much in it. LLMs automate it like never before.
Plain advertising, governments' propaganda, political propaganda for one group or another to shift public opinion (it's done on TV networks, why would they not do online campaigns?), astroturfing by corporations promoting acceptance or fighting negative news (e.g. rideshare, AI, whatever certain wealthy personalities are doing) ... the list goes on.
HN has always been relatively influential in the tech industry and therefore worth influencing, and now the cost is very cheap - you don't even need to hire many people, so less-resourced operators will find it worthwhile (and they will also attack lower-value forums).
If you farm a fleet of good accounts, you control the discourse. On HN, you could boost whatever you're trying to push, and downvote or flagkill whoever objects.
There are obvious benefits to controlling public discourse, right? Even if it's just to support some project you're working on.
I've been more disturbed by comments that were flagkilled just for being wrongthink, not because they were rude or not well argued. I've also seen a lot less of those flagkills over the last 6 months, which makes me feel like there were some fake accounts that got caught and culled.
In the recent thread about life in a class war, a lot of comments in different places saying that if we don't fix this inequality problem, g-tines might come back, and every single one of them was flagkilled, no matter whether it framed as "we have to get out the g-tines" or "we have to fix this, otherwise psychopaths will get out the g-tines" or "thank god we've become civilized enough that we don't get the g-tines out"
Set text size as preferred, underline links (or not), turn off display name styles (or not), ui density compact or default, chat message display to compact, space between message groups 0px, turn off all the animated emojis and gif animation stuff if you want.
In client use, there's a button to hide member list (or not).
You can definitely make discord look like a slightly less dense IRC client (mainly because of the channel picker) if you want. And if you want to go really crazy use it in a browser and userscript customize it or use betterdiscord.
I think a lot of the features like embeds and emoji reactions add a lot of value compared to IRC (which I think is also why the IRC world is trying to add those features).
Sort of, except if no one can ever discover a community it is always dying by default
Personally I'd love to find a decent online community these days, my social circle has shrunk considerably, but idk. It seems difficult to start fresh with new people nowadays
we were made to socialize in person. you can mimic it online and nourish existing connections over it but nothing helps build friendship more than being in the same place at the same time a few different times and talking to each other
Thats true but online content has always had its place. 25 years ago finding forums and irc was a god send, my lonely hobbies and interests became things i could regularly talk about. Its just modern social media abused the system, the algorithm, and us.
Which is all to say i agree about needing mostly irl, but there is also something of online community that irl could never replicate (for most people).
on the public servers yeah. but the ones im in with real people who know each other will be fine.
I think the problem is not keeping agents out of private real people spaces, but for people who dont have any pre-existing or 'real world' connections to these communities to find a way to prove they are a real person over the internet alone and get an invite
On a related note, I think this is going to be the biggest challenge to most folks when it comes in resisting using government ID online. it will be the apple offered for easy proof youre not a bot to normal circles.
Discord is terrible. Full of bots, creeps and ai slopped to the gills.
Some communities are better than others but the sheer volume of stinky trash is immense despite discord and the poor volunteer moderators efforts to prevent it. Most mods are neutral on it too.
There are chat communities that are still somewhat safe with zero user verification. But I will not mention them.
There's this old meme where someone asks what will happen when AI bots posts helpful, curious and thoughtful messages!? That's mission accomplish :D They can't be better then the average human though because of training data, so I don't worry about AI comments getting up-voted by real humans, I am however worried about fake upvotes.
Why do you think those comments are accurate? Maybe those comments are by LLMs? If you believe crowd wisdom on its face, you will have big problems with LLMs.
Reddit sold it's data to AI companies for training[1]. They could have refused, but companies like OpenAI likely would have harvested that data anyways. As such, it should not be surprising that AI models are pretty good at generating reddit posts. They were specifically trained to do that.
This is sad, because Reddit remained one of the final bastions of human content on the internet. For several years, appending "site:reddit.com" to a google search was a valid way to get something usable out of a google search. Doing that is still an improvement over raw-dogging Google's ranking algorithms with an unfettered search, but AI slop increasingly is the result.
This is one of my great disappointments in the current rise of AI. LLM's can give good search results when dealing with a topic they've been specifically trained on by human experts, but they're not good at separating human-produced signal from AI slop noise. We've done nothing to prevent a sea of AI slop from being dumped on top all the human signal that's out there. When AI companies enter their enshittification phase and stop investing in expert human trainers, the search results LLM's produce are going to fall off a cliff. Search is a bigger problem than ever.
Communities in FB, WhatsApp, Telegram etc are actually flourishing. As it appears real time gated communities are doing fine.
It’s an unpopular opinion but I am looking forward to ID and age verified social media. If done right we can have real people around again.
BTW, ironically the harsher communities like 4Chan doesn’t seem to suffer from the dead internet. I guess it’s either because the advertising value is too low to justify AI use there or maybe AI API providers refuse to work with such a content this reducing opportunities to infest with bots.
> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs
People using LLMs without being fed their own post history are still pretty easy to detect. There's just something very recognizable about the cadence and tone of LLMs.
What really stuns me is that if you call someone out for it, 9/10 times you get absolutely buried in downvotes. Even here on HN. Its like people are angry that you're lifting the curtain on the slop, that the writing they enjoyed is fake.
Reddit astroturfing firms and bot farms learned to buy/use “seasoned” accounts over a decade ago. I’d venture there have been countless bots just in a holding pattern harmlessly building up reputation and a human-like history of posts across different subs etc just to eventually be either activated or sold to someone else to “burn”
It used to be super common that when you spotted a bot post and clicked through to the user's history, you'd see very average, human-looking activity from years ago, followed by a long gap of inactivity, and then a flurry of obvious bot comments.
It's very obvious that these accounts were abandoned and then either bought from their original owners, or more likely bought from someone who compromised them, because of their history and karma.
And I would bet money that Reddit is well aware of this phenomenon, because not long after it became so common as to be impossible to ignore, they papered over it by allowing users to hide their history from public view. (AFAIK subreddit moderators can still see it, but typical users now have much less ability to see whether they're interacting with actual humans.)
I recently spotted one unmistakable example of this[0]. It’s been a trick for many years now that duplicating a human post and its comments is a good way to appear human but this was quite the example.
> duplicating a human post and its comments is a good way to appear human
Also just repeating something from the linked article, but often with different wording and in a tone that makes it seem like it was something that the article missed.
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
You can have both IRL and online-free-of-bots. I already wrote about it but one of the very best forum I'm a member of, where real people are posting, requires to be vetted in, web-of-trust (but IRL) style. It's a forum about cars from one fancy brand and you can only ever join the forum by having a member (I think it may be two, don't remember) who's already in confirm that he saw you driving a car of that brand. It's not 100% foolproof (someone could be renting the car for two hours and show up at a cars&coffee or take a friend's car etc.) but this place really feels like a forum of yore.
And people do eventually travel, so it's bound to happen that an owner shall go to another country, meet someone there, vet him in etc.
Now, sure, it may not be the "1 million users acquired in three days thanks to my vibe-coded app" scenario but that is the point.
You can imagine other domains where IRL communities have local groups, but where forums regroup different IRL communities all interested by the same hobby/topic/domain. And when people travel and meet, the vetted members do grow and connect.
Oh and on the forums a lot of the posts are pictures, where "Julian xxx" met "Black yyy Cyril" and you see both cars (and from more than two people): suddenly it becomes much harder to fake a persona... You now need to fake both Julian xxx and Black yyy Cyril and fake the pics. And explain why your car has never been posted by any carspotter on autogespot etc.
You can imagine the same for, say, model trains: "Met Jean at the zzz meetup, where he brought his wonderful 4-8-8-4 'big boy' locomotive, I confirm he's into the hobby, vet him in".
Naysayers and depressive people are going say it cannot work but I'm literally on one such forum and it just works.
P.S: if I'm not mistaken in the past in some nobility circles you had to be vetted by up to sixteen (!) other people from the nobility that'd confirm they knew you, your parents, etc. before you'd even meet the king/emperor/monarch to make sure that someone from far away couldn't come to, say, Versailles or Schonnbrun pretending to be a baroness or count or whatever. Quite the extensive check if you ask me.
I find it amusing that this is the top comment. Reddit is so awful you finally wrote it off, but not before you used it to try to “karma farm and do some covert advertising”. It’s on-brand for HN hypocritical bullshit. But, since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard, have an upboat fellow traveler.
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
Good. Tech has been artificially propped up to generate liquidity in markets. Never had organic demand or traction. Most output was trite and meaningless; just capitalism.
Zero cares if all the HNers rich on tech stocks go broke.
Happy to project as little concern for their well being as this forum has projected at other's fields tech eliminated.
Some wild Dark Triad shit peddled here where there is no obligation to humanity's problems, just HNers own personal preferences.
Yeah ok that can be projected back, terminally online script kiddie; fuck your stock portfolio and I hope you end up in public housing assistance.
I feel you. Especially in the larger subreddita. i participate, and mod, a few small ones, and the community there is pretty strong and folks shut down ai slop pretty quickly.
I'm not saying being a mod means it's bullet proof, but i do notice smaller communities tend to self police better and know what's real.
This kind of thing made me imagine the creation of "digital towns" the other day.
Imagine an online community where you can only join on the recommendation of two other members, who you must have actually met in person, to participate. Meanwhile, you leave at least some of the activity publicly available to the general public so that interested parties can meet up IRL and join.
This could probably be implemented easily on top of existing online platforms like Discord, Reddit, etc. since it's really just a community building rule, not a community itself.
More of a philosophical question but if you have no idea whether it's a human or robot, does it really matter? Personally I dislike AI slop only when I can tell it is...
- I am trying to learn about the topic at hand and trust a human's comment more than an LLM's guess
- I am trying to connect with other humans to fulfill my social needs
- I am maybe spending time to help another human out with a response because I want to help someone else
- I am interested in the perspective of other humans
Those are just a few reasons. For each of those if it's actually an AI I feel I'm losing out on something.
It's easy to botspam Reddit because even the real users always acted like bots. The big subreddits were the worst, but contrary to how the users keep saying "it's good if you find the right subs," no it's not. Wrote that place off like 10 years ago.
It might come down to shareholder/IPO stuff but you can tell Reddit doesn't actually care to put the effort in to crack down on bots (however you'd do that) because they already don't give communities proper moderation tools/third party tools and the site does censor
Whatever allegiances (with people, or allegiances to ideas) Steve Huffman has, or people like him - it's not enough. It's a site seemingly killed by greed
(Yes, I know moderating this stuff at scale is hard)
It used the browser agent to grab user cookies after signing in, then made API calls iirc.
Using just a browser is way too token intensive and slow. It would look for 401 errors then run the browser automation to login with the credentials and grab the token.
I'm surprised these platforms don't have advanced heuristics to detect API calls and inauthentic traffic.
Did you clone the Reddit API from browser traffic and then turn it into a 100% API driven thing?
I'd imagine they'd be sniffing browser agents, plugins, cookies, etc. to fingerprint. Using JavaScript scroll position, browsing rate and patterns, etc.
Maybe their protections just aren't that sophisticated.
People still see HN as a valuable place for promoting their businesses. Usually some poorly thought out SaaS.
If you look at what people outside HN talk about HN, it's not uncommon to see wannabe tech entrepreneurs talking about how to promote their products via Show HNs and how to stay HN front page. It's honestly a little sad considering that HN has a tendency to rip these projects apart.
the suits or suit minded people have realised that HN is good for advertising to the kind of demographic that'll give them free labour and is easily swayed by whatever the latest trend is
> HN's guidelines and tone policing are more easily followed by a bot than a human.
HN's guidelines aren't that strict and the mod hammer is a plushie. It's not difficult to get by here. It's also kind of useful for critical reflection/self-regulation to hear the occasional "you came in too hot" or "don't be boring" from a moderator.
Seems better to me to just try to be sort of reasonable and let the mods nudge you if they need to and let your comments be downvoted from time to time. What is the goal of these people, to never experience correction in their lives? To never write an unpopular comment?
He's stating a fact. Turn on showed in your options and scroll to the bottom of the comments on any popular story. There are so many agentic users here.
I just don’t even understand the appeal of having a bot interact on forums for you unless you’re astroturfing for your company or personal brand or whatever
This site is CLEARLY astroturfed to hell and back and infested with bots. Any attempted discussion of this fact gets killed REALLY fast.
This part of the guidelines is a 15 year out-of-date bad joke:
> Please don't post insinuations about astroturfing, shilling, brigading,
> foreign agents, and the like. It degrades discussion and is usually
> mistaken. If you're worried about abuse, email hn@ycombinator.com and
> we'll look at the data.
"We'll look at the data". Sure buddy. You'll do what you always do, which is apply to banhammer to anyone that's not following your talking points, and tone police the actual humans.
Enjoy "conversing curiously" with bots while the mods tone-police non-bots out of existence.
For what it's worth the admins here have let the tone of conversation slip a little when it comes to AI, as in there are many people who now openly mock (and worse) the AI zealots and there's no admin coming in and "saving" the metaphorical day anymore. In the not so distant past that kind of behaviour was almost instantly reprimanded, kindergarten-style.
Exactly, they aren't good at creating new material. But many discussions in comment section are simply regurgitations of existing material, which they are good at rearranging. New novel discussions in places like this are actually a very rare thing, as many comment sections are simply people who already know informing those who don't. I'm doing that right now, funnily enough.
No, they aren't even good at rearranging existing material. They produce bad writing that only superficially looks good in a lowest-common-denominator sense, and falls apart under any close examination. Everything is wrong with it, from the sentence structure to the rhetorical forms to the substance. AI 'writing' is a loose collection of cheap tricks that score well on A/B.
It’s poor logic, a non sequitur. An absurd reduction. By your argument anyone who hasn’t written a great literary work is a poor writer, and would be bad at writing online comments.
LLMs aren’t lacking in the sort of writing skills that make for superficially good content. They know grammar, they know rhetoric, and they know their audience. You can’t tell them from a human on their writing skills. Where they tend to fall down is their logic and reasoning skills, and unfortunately it seems you can’t use that to distinguish them from the average online opinionator either.
With the current batch of SOTA models, it is not hard to prompt a model to pass the sniff test on social media forums. If you don't believe me, try it.
All you really need to do is give it some guidelines of a style to follow and styles to avoid. There's also a bunch of skills people have already written to accomplish this.
I have worked with LLMs for a couple years at a very non-technical level and it was not that difficult to give it proper prompting and reference material.
If you are reading LLM content just about everywhere and have no idea. Obviously there are easy to spot things, but the stuff you don't spot is the stuff you don't spot
People that like to fancy themselves as good llm content detectors just end up accusing everything they don't like as llm content.
The only thing worst than a slop comment are the people that bitch about it incessantly. I'm convinced it's become a new expression of a mental illness.
The main thing I suspect of being LLM written is the sort of LinkedIn style: very short sentences, overly focused on sort of… making an impact on the user. But that’s also how a certain type of bad human writer writes. So in the end, I’m not sure I know if anything in particular was written by an LLM.
I guess… “that’s not just an AI red flag, it’s generally shit prose” would be how ChatGPT would describe most things nowadays.
It’s the distilled mediocrity of the statements. Never venturing beyond a 10% margin of what you would get if you sampled the opinions of 1,000 people who underwent jury selection by west coast liberals.
Was that written by an LLM? It isn't that it's a mere opinion, it's that when every word out there has to be scrutinized for the possibility that an AI output it instead of a human intelligence that it gets pathological. Am I an LLM with the right prompts set up to respond this way? I mean, I know I'm not, but everyone else out there is just going to have to trust me that I'm not.
On the other hand, I’ve been accused of being AI/bot and if I say things the mod doesn’t like and is not their favorite thing to hear I’m “flamebaiting” or engaging in personal attacks when pointing out specific things.
Frankly, online communities have been doing for many years now, when the censorship, anti-free-speech, tone policing mods and mobs started dominating online and America really did not have the self-respect or confidence anymore to enforce the Constitution online.
I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political". This is somewhat related to the Overton window but really a bunch of (mostly conservative) ideas get normalized so aren't deemed "political".
I see the same thing with "AI Slop". Yes, there is AI Slop but (IME) it's pretty easy to spot. But what's more annoying is how often people are willing to throw that accusation whenever someone takes a position they don't like, much like the "political" label. It's lazy and honestly just as bad as the slop itself because it unintentionally launders the slop in a "boy who cried wolf" kind of way.
I also have a theory that some AI slop isn't inherently successful. It's just heavily botted by people who are interested in promoting certain positions. I bet you could make a pro-administration LLM bot and another one promoting a communist revolution and no amount of model tuning would make the second as popular as the first because the first would hit third-party botting as well as platform content biases (eg Twitter).
I've personally been accused of being a bot. This is particularly true in recent time as I've tried to share facts and fact-based analysis of, say, what's going on with crude oil markets, the military operation in the Gulf and the politics and economics around it. I even saw one hilarious comment saying (paraphrased) "the bots are getting clever and posting about unrelated topics". This was funny because it never occurred to this person that no, it was just a real person posting something you disagreed with.
> I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political".
This happens on HN all the time. For a lot of downvoters and flaggers, there are two kinds of opinions: "Things I agree with" and "Too political for HN."
> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.
This just makes me wonder...so what?
Some of the oldest posters here with the most karma continue to post absolute garbage takes on topics ranging from US healthcare to history of USSR, that are trivially disproven by learning the very basics from a Wiki article (e.g. not a high bar).
To be fair, this opinion slop is also present for new users and LLM bots, but is one kind really worse than the other, if both of them contribute to killing the community?
We already know what kills communities. It's the eternal Septembers. Infighting within leadership also doesn't help, but time and time again it's the influx of too many new users that nosedive and drown out quality contributions.
Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”, and then actually spend time reading them?
No? I’m imagining not at least. Because there would be no point to it.
If you would enjoy it, then I’m surprised you’re here and not just simulating the experience with your LLM by yourself.
> Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”
The reason I'm not simulating the experience with an LLM is because:
1. It costs more time to do so, because I have to prompt it to create a single comment. Multiply that by the typical number of an HN thread.
2. I suppose in a way you need bad takes to form your own view of a topic or an issue. LLMs would also be unable to provide truly unique experiences, such as some of the veterans who sometimes post here who were part of the living computing history as we know it.
> I’m surprised you’re here and not just simulating the experience with your LLM by yourself.
That's something you imagined that I claimed I want. If you read my comment again, you'll see there was no such thing.
An irascible human being with "wrong" opinions is still better than a polite and factually correct bot because there's no fucking point in having a conversation with a bot. We're here to have conversations with people, not to prove fact beyond a reasonable doubt.
Do you really not care one way or the other? Would you really rather just be talking to LLMs here? Or would you just script yourself as well and call it a day? Then what?
> We're here to have conversations with people, not to prove fact beyond a reasonable doubt.
Maybe you are. I like getting to a reasonably correct model of a topic or issue. Bad human takes can still be useful here. I just get inevitably tired of the people crying about potential LLM comments all the time.
> Would you really rather just be talking to LLMs here?
Obviously we're not there yet, regardless of what I want. But there is a great number of HN threads posted here that touch on topics that have been discussed so many countless times, that an average LLM summary would do better than most comments.
If you head to Twitter right now, the vast majority of bots are blue checks. It seems to actually encourage the opposite, where you trusting that someone paying $8 for an account makes you even more likely to fall for slop
I think twitter is an odd-one-out here, twitter as a whole has been heading down hill ever since the acquisition, and I wouldn't be surprised if many of those blue checks are officially sanctioned bots. Especially given the way so many of them push the same narratives that Musk does at the same time he does.
Unlike a lot of communities, yours at least started on the correct side. Better to ban outright, than to slowly realize that you should have banned it.
Indeed. Take a soft approach, or "wait and see", and you'll just allow your community to get infested with slop enthusiast crybullies that loudly protest any pushback against "genai content". The communities that draw a firm line and hold it will be the only ones that endure.
It was a surprise to us how vehemently some folk defended AI content and assumed it was their right to post it within our community.
We had no problems with people using it and posting elsewhere, it was the demands that we must allow it that were problematic and made us question whether we were doing the right thing.
No regrets now, though, as we see competitors being flooded with AI slop and they are too invested in it to change now.
>shrug off around 600 AI content creator accounts monthly.
>I fear losing the battle.
I was in a small niche creative writing community for a while. Circa 2021\22. AI wasn't why I was there but I demo'd a few LLMs to a lot of the users in the Off Topic section because people were curious. Even with an explanation of how they operated, almost everyone was at least interested. One author told me how he operated similarly, rote learning how to write like his favorite authors by copying out their texts, hand written, word for word. Their concern was largely that they were too hard to use from a technical perspective.
These people knew I was there to learn, and that I was unlikely to ever try and publish LLM derived content. I said as much often.
Sometime in late 2022, a switch was flipped. And almost all of them started talking about how AI and those who used it were unambiguously evil. They didn't say my name, but they stopped engaging with me. Gradually, they started reposting twitter content from extremely anti AI people. Complained about AI submissions to various publications. Eventually, someone reposted a tweet calling for the death of anyone who used an LLM, with not even a single disagreement (and lots of encouragement)
I just bailed. I had only ever engaged positively, answered questions for the curious and tried to help people out. I posted one AI assisted story, and that was to demonstrate how my contributions were tracked vs AI contributions automatically in the editor to satisfy someones curiosity. Clearly highlighting the bits I had written. Just a technical demo. No one was asked to enjoy or positively engage with it as if it was human written.
A while later, most of their submission rules were updated with a new clause, if it was judged that AI written content was discovered, they would blacklist that person from all submissions across their entire community. Considering I had demo'd LLMs, and the uselessness of AI detectors, it was clear to me that these people would be able to justify blacklisting me if I poked my head up at all. I had been developing my own story for submission (myself, no LLM content), but I just dropped it. I just didn't feel like sticking my neck out for the witch hunt.
I also used to be quite engaged with blockchain. And it went through a similar process, most people ignored it until that paper about the power usage (Claiming it would spike to some level it never reached) and then suddenly being associated with it was an outrageous moral crime. But after a while, when it turned out that the power use claims were largely a nothing burger, people gave up on the hate parade.
I don't think you will "Lose the battle" (at least in terms of keeping AI users out). And its always ok for small communities to be selective about their membership. I just don't think its possible to maintain such artificial rage for more than a few years. The AI Datacenter water/power claims are a clear London Horse Manure problem that looks set to resolve itself, and the copyright issues will get sorted to some degree. Eventually I think you just wont care enough to ban anyone except low effort spammers (of which there are a huge amount, granted).
I kind feel this might be good.
Bot writen comments and AI media that can no longer be distinguish from real, will make us human leave the social networks, which helped to separate Us humans.
Going back to the real world were you can trully believe on what you see, and enjoy the tone, look and scent of of our fellows humans beings.
This seems naive. As long as people are "enjoying" the AI-infested social networks, or at least not annoyed enough to leave, they will stay on them, and become further disconnected from reality. We have half of EU teenagers talking to chatbots regularly. Alienated people flock to them.
Social media that caters to what the user wants/interacts with can become infinitely more so. This is already applied to entertainment content across tv and the internet.
At some point an instagram/tiktok/etc user could see nothing by real people and not even know what is promoted vs ad vs post.
A few tech companies managed to get massive numbers of people addicted to toxic social media content that was terrible for mental health but made a small group very wealthy. I don't think those same businesses and execs are just going to pack up and go home with an even more powerful content tool available now. LLMs are going to be used to create skinner boxes that make Facebook and Twitter seem like wholesome communities.
The problem is that many of us have niche interests and no one local to discuss things with or get made fun of for being a nerd.
I loved maps and geography as a child and still do. I've never met anyone in real life that likes it as much as me. But on the internet there are places were I can discuss it and other people share fascinating articles, pictures, etc.
Problem at scale. Doesn't matter if someone is consciously able to identify individual bot accounts or comments. There can still be a strong general feeling that something is very wrong. Leading to more and more frustration and unhappiness.
"Popular" reddit posts and subreddits are a good example of this.
One of the paradoxical things that makes me hopeful is that there's going to be such an incredible amount of low effort AI slop content that it's going to drown out the low effort human-made content and generate a large amount of distaste for it. So much will be so bad that good taste and high quality will be rewarded more status as the people who will say and believe anything will be led astray and left behind.
Maybe it's hard getting across what I mean so a more concrete example is there will be SO MUCH clickbait out there that serious outfits instead of being forced to do it will be able to successfully differentiate themselves by NOT doing it. (and many similar things in different arenas)
I'm trying to say that LLMs raising the noise floor will drown out a lot of the toxic noise that's been plaguing us.
> So much will be so bad that good taste and high quality will be rewarded more status as the people who will say and believe anything will be led astray and left behind.
I really want to believe this will be true. However, I also suspect there's some external driving force, that I cannot readily name, which is making people incapable of consuming anything except this low-effort content. I mean, obviously it's working to some extent. Perhaps AI will be the thing that accelerates its death, but part of me thinks something else needs to happen beyond just an increase in useless content.
In my opinion there isn't an external _nefarious_ force causing all of this. Certainly those forces exist but without them much the same would be happening.
It's the economy of everything being free but supported with advertising. That mechanic is what leads to the race to the bottom lowest common denominator human motivation hacking attention toxicity. (yes that's a bit of a ramble).
If people weren't getting paid for the smallest increment of attention they could grab, it wouldn't be promoted the way it is. I don't have a high opinion of the things which grab my attention, but they still manage to do it sometimes. I think many people are in that boat. If there were other mechanisms with which we rewarded people for doing things, something different would be optimized.
And people just wouldn't reward the 10-second-gratification in anywhere near the same way if it weren't for the advertising.
The important thing to recognize is that quality of content has never been the driver of online communities. As long as they provide an engaging break from real life, they will exist and thrive. I think the negative association with LLMs is a phenomenon that will die out in the 20s. Our understanding of authenticity will evolve and so will the tools and platforms. The internet has always been extremely artificial, that won't change very much.
The balance is so far out of whack with LLM's now in online communities. People crave human interaction with like minded individuals, and whoever figures out how to give authentic online experiences is going to be successful. Maybe small communities need to come back, where you build credibility slowly. Why does every site have to be a monstrosity that wants to build a hundred million users to IPO. It just attracts the worst. I was active on Reddit for years under the same username I have here. I have pretty much abandoned it.
> People crave human interaction with like minded individuals
I don’t think they crave it enough to make a difference. Even before AI slop, Reddit had made successive changes that led to much less of a feeling of interaction with real, authentic humans who could become your buddies. The UI de-emphasized usernames and hid the sidebars where subreddits could have their own distinct community atmosphere. I hear that now on comment threads, Reddit will even hide a decent number of posts from other users, so that a poster may well be talking into the void.
It is on old-school fora that one can get a sense of actual interaction: with avatars and other personalized touches it’s easy to gradually learn who is who, and there is a culture of longform text where you can actually get a sense of other people’s personalities. But how many people under the age of 35 or 40 are joining those fora that survive? Give people a choice, and it turns out they prefer the dopamine hits of engagement-maximizing commercial platforms, and the smartphone as the default (or sole) interface to the internet with all the death of nuance that spells.
Some definitely enjoy the dopamine hits and get addicted to the doom scrolling. Maybe I am just too old to understand it and the internet is passing me by. Some of us still like conversations like this. Real conversation in a respectful manner even when we question each others viewpoints. The old internet is still there in some places and I'll continue hanging out there as long as it does. While I have great friends in real life, not that many of them are old tech nerds, so the internet is really the only place to talk to like minded people.
The company I work for has a deep rooted community side and despite what big techs do, I am 100% confident the only aspects we have in community features are for the user benefit. No gray area. Just that.
Since the AI sloppification we lost considerable amount of traffic to bots. But worse than that, we lost users who tended to contribute back with others.
We can leverage multiple ways of exposing community data to members, so it is not that we are loss because of that, but more in the fact that we have 30y or so of good feedback on how the community around the platform was good for people and now everything is at risk...
Don't get me wrong, my work is work... There are premium features and else, but the amount of value one can get for free is what the platform is known for. And we know many people use it for free for years and when they need or can they subscribe and mostly stay for years and years.
The fact people are losing those connections is depressing to me
I left multiple online communities because the slop and the slop users were unbearable.
I use ai okay. I think it's useful. But people who dove hard into this stuff treat all text on their screen like it's a chat bot and not a person.
"Rewrite this code using the new API" "excuse me?" "Can you do it I need it right now chatgpt won't compile!" "Show me your code please" provides the biggest pile of dookie ever "hey can I ask how you came to decide on any of this? Maybe we should rewrite what you have here because x y z is concerning" "the ai did it I am learning. There is no need to rewrite anything just write this section for me" " no thanks" someone else does . user leaves
It adds a depth of nuisance that's for sure. I've seen users talk about how they can't wait until they don't need to ask for help anymore and can just use LLMs. Meanwhile I'm directly messaging with the person who made a package and asking why they designed it the way they did beyond greatful to learn 8 new things in 6 sentences.
Sadly the imperative is, as often, a call to everyone to be good guy and make less noise. Unfortunately, it doesn't work, neither at personal level, nor at global.
One may be quiet, but what if your friend/acquaintance/fellow got possessed by some AI slot machine, and is sharing his "products" enthusiastically? I had such case, and right from the very beginning was dismissive and rude, and it doesn't work -- he keeps sharing various artifacts.
On a global level, yes communities die out. I think, global communication has reached the point when it's more a liability than a benefit. In late '90s and early '00s, maybe until early '10s, getting more connected could lead you to nice clients, getting hired etc. Nowadays, even before ChatGPT 3 in '22, every such area became overcrowded, underbidded, etc, and LLMs, surprisingly, added not much new -- just augmented this trend.
Giant online communities, yes. Small ones seem totally unaffected afaict - some harder to spot scam/spam accounts, but they're outed as soon as they act. And any invitation-based thing should almost perfectly block those.
Smaller communities are generally a lot healthier anyway, so tbh I don't think this is all that bad of a thing. I don't think it's possible to be open to millions and also be healthy, unless you spend a lot of money paying moderators (and regularly rotating them, to prevent burn-out or mental harm from too much exposure, which ~0 do in an even slightly ethical way).
A few things. A web of trust of some kind like vouching may come back, and general algorithmic silencing of low quality members. Also most governments are going towards the South Korean model of government-verified ID to post online to keep teenagers off social media. The same tool can be used to greatly reduce spam and slop, if that's what platforms want.
Also people will get used to AI in online spaces as AI quality improves. If I'm online trying to get help for some task, I personally don't care who wrote what if it is correct; it's not like humans have great track records of accuracy or substantial contributions either on average. Correctness is expensive in general.
If I'm online trying to relate to other humans emotionally, well I get what I'm paying for. It's been true forever that the better the gate, the better the community. I've tried to push the boundaries of openness, but as I've written extensively on MeatballWiki, soft security depends on there being more good than bad apples in a community. With machine intelligence, the economics of that are silly.
Regardless, people love people, so we'll figure it out. I'm optimistic we can rise to this challenge.
Entering the AI era, it's hard to tell the authenticity of things on the Internet. But sometimes, having a conversation with AI is not a big deal as long as we can gain something from it
There has to be room for an AI-driven project that expresses a unique idea, even if there's no community around it yet. Someone has to express it, and from now on that idea will largely be implemented with AI.
> A good use of AI is when it enables people to do something they couldn’t do before, to contribute to a community when they couldn’t before.
I agree 100% with the novel contribution aspect. But there's some nuance there.
For example a project might have no active contributors. It might not be something you can drop directly into your codebase. Neither of those is inherently bad.
As AI becomes more responsible for higher-level planning decisions, the value of an OSS project becomes less tied to visible community activity like PRs and issues.
I notice this in my own work a lot. I might not use that project's code directly. But I think about a problem differently as a result. I often point my agent to existing OSS projects as inspiration on how to solve a problem. The project provides indirect value by supporting architectural decisions, deployment approaches etc. Unfortunately OSS activity doesn't capture this.
> A good use of AI is when it enables people to do something they couldn’t do before, to contribute to a community when they couldn’t before.
There are two separate things here that are getting silently conflated.
> A good use of AI is when it enables people to do something they couldn’t do before
This could be good on an individual level, if say, a doctor wants to vibe code an app of some sort for his individual practice.
>to contribute to a community when they couldn’t before.
This is where it goes off the rails. If they couldn't meaningfully contribute before, they aren't going to suddenly be able to discern that whatever slop they want to contribute is of value to the community. That's just another way of saying, if I wanted an AI opinion on something, why wouldn't I get it directly from the source, and write the prompt myself, instead of have some intermediate human prompt the AI for me?
The human has unique context. They may work in a niche domain or they talked to people and observed an unsolved problem. Then they express a potential solution via OSS. It's like product sense. Then they share that with others who find it interesting. The code is a great way to encapsulate the idea. It is usually the result of research and back and forth not a single prompt. It would be way harder to think through or build a solution without AI even if they had context.
Because of the convenience. Why should I have to go and spend my time prompting an AI, if someone else has already done that for me. Same thing with food. I know how to cook a chicken risotto, but sometimes I like having someone else do it for me.
Who is going to verify that an AI-driven project is a unique idea? How do you distinguish between a genuinely unique project, a grifter who is shilling their "unique" project, and a new enthusiast who is convinced their project is unique, but is not? This is an impossible moderation task. The only options I see for a community are to either totally ban AI-generated content, or be totally consumed by it.
I don't really know. Certainly we need a higher bar. The Kafka example in the post may be hyperbolic but I agree it pollutes the space. But we also can't swing the other way and rely completely on out of date proxies. If you ban AI code there will be very little code to see in a year. It'll take time but we'll arrive at new norms. We built semi successful ways to filter content farms in the earlier internet days. The signal has to shift to "did they think hard about this problem" which has some observable properties. Like how they articulate the problem, or why it became important to them.
I have pondered the sensibility of using AI to support the initial birth of new communities. Given the needed social validation of seeing both 1. A populated community and 2. The tone of the community being grounded, non-toxic and useful.
The alternative is having a community born that will be small, have early adopters who can be overly passionate or critical and gatekeep folks from discussion. That means high effort to curate initially.
The “slopification” of the internet has been happening for years now but I honestly don’t know what a real solution would look like.
Most people aren’t willing to go through a identity verification process, or pay to join a community, and invitation-only spaces would probably lose diversity of thought pretty quickly.
Even still, I guess one of the above is a lesser evil because the bot problem is only going to become more unbearable.
P.S. Props to the author. I really liked this writing style.
I think one of the big differentiators of slop is critically reviewing the work as a human and continuing to refine before letting it see the electrons of public places. I think that the true benefit is harnessing AI to augment ourselves as an extension of us, rather than a replacement. I'd be curious if there is a way to effectively prove something is human first rather than AI first on the internet. I haven't figured out any particular way yet as even using AI to detect AI would require a sufficiently large sample to determine. Something that keeps me awake at night.
When LLMs were new on the scene, I thought trust would fade in the written(text) medium. I saw it happening on Substack, Medium, and Reddit. But then VCs pumped so much money and AI has gotten into every other modality (audio, video). The only thing I really interact these days are the human beings sitting in front me, phone calls with people I know and hackernews. Life seems sorted but something feels missing as well.
Edit - I am not anti AI but it is slowly killing the digital human interaction.
The sad part is that the cost gets pushed onto the good participants. Once enough replies feel synthetic, real people spend more energy deciding whether the conversation is worth joining.
Wow, that's bad. Looks like the warnings about TPM and remote attestation being a backdoor to total digital lockdown from the Stallman contingent were right.
I was on Usenet starting in 1991. Once the Internet got popular with the general public around 1995 things started going downhill. Spam overwhelmed Usenet in the late 1990's and made it almost unusable for general discussion.
Stuff started moving to web site forums which I still don't think are as good as a Usenet newsreader. slrn was my favorite.
Then reddit came along and a lot of online forums started dying as people moved to reddit.
Just this morning on reddit I reported 4 separate posts as AI slop to the moderators. They need to change the categories as I flag it as "disruptive use of bots"
For 2 of the posts the moderators agreed with me and about 5 hours later the posts were removed. For the other 2 the moderators haven't done anything.
It's a losing battle.
Some of the posts start by asking questions like "I was thinking about this and... [long rambling paragraphs] Your thoughts on this?"
I waste a minute reading then another minute skimming the rest of it and then realize I wasted 2 minutes of my life. Then another 30 seconds reporting it to the mods.
This has exploded in the last 6 months.
Then there are all the repost bots farming for karma. Some subs have a rule that you can't repost something in the last 30 days or 6 months. But it is really ridiculous when something get 500 upvotes and then literally the next day a bot reposts the same thing and it still gets 300 upvotes. I think it is just a bot farm upvoting stuff.
The baseline level of trust in an online interaction has been eroded significantly by LLMs.
The question is, how can we reverse this trend and increase trust?
I have a sneaking suspicion that it would help enormously if the stock prices of the largest companies in the world were not tied to how effective they are at hijacking as much of humanity’s time and attention as possible.
Maybe the fediverse can (eventually) help? It’s been a while since I looked at it.
Let’s empower people to effectively have more control over the content they interact with.
Social dynamics can make this difficult. We all want to be in the loop. The recent striking successes of the movement to ban phones in schools gives me hope.
> Maybe the fediverse can (eventually) help? It’s been a while since I looked at it.
The fediverse has been around for well over a decade in some form or another. It never caught on with society enough to make a difference. And unfortunately, the fediverse has now developed such a distinct culture of its own, Highly Online people with distinctive political and social shibboleths, that it even alienates many tech idealists around the world, let alone the general public.
The general public isn't alienated from the fediverse because of its distinctive political and social shibboleths, the general public simply doesn't know that it exists.
As far as the "tech idealists," a lot of them seem to want every space to be 4chan where they can be racist trolling assholes without consequence. And those folks have Nostr.
I think what could work is requiring users to prove their authenticity and uniqueness using a national ID of some sort. It would be bad for privacy, no doubt, but it surely would work. But the users' actual names should not be displayed.
I was thinking about that. It should be possible to do this in a way that mostly preserves privacy.
Sites and apps don’t need your actual national ID, just to know that you have one. I think it could be possible to have 3rd party verification services that don’t know where the verification request is coming from, thus preserving privacy on both sides.
I feel the root of the problem is that Google and major platforms defined "correctness" as "high impressions" and "high engagement." This created a game where AI-generated "slop" becomes the ultimate winner. For those of us trying to create or find constructive, deeply-thought-out content, the situation is becoming increasingly dire.
It is exhausting to see a single, sincere sentence based on genuine human experience buried under 1,000 pages of SEO-optimized, AI-generated "void" that Google deems "correct." Despite this, I will keep working on filtering through the noise today.
I feel that a lot in my side projects: maybe one should keep the half-baked AI repo for oneself and rather share what the experiment, the thesis, and the learning from the building are. No one cares much about the (un)finished product, as it can be replicated better in most cases with a couple hours' work of claude coding.
For instance, I really liked how Karpathy shared a high-level idea on the LLM-based wiki. It was sadly followed by a long tail of no-one-cares-about "Here is my LLM wiki product" posts pointing to the generic LLM-generated landing page.
I want my future community apps and sites to build in bot a flagger. I don't care how hard it is, the community that gets this right is the one I'll jump ship to.
It sucks that the narrative framing device of 'human slop' has vanished in the last year. Some subreddits, like all location subreddits, lifestyle subreddits like malefashionadvice and redscarepod and entry-level academic subreddits like math and criticaltheory were already just hives of human slop before AI came around because of a structural design to the site that had the side effect of normalising a total absence of quality control.
Upvotes are not a good mechanism for quality control in any way because they force good content to have the same metadata as the content that is technically well-constructed but is irrelevant, meaningless, just a platitude, too obvious to be obvious or pablum. Upvotes turn everything into a shock-value dominated 101 space.
Online communities that allow upvoting / downvoting have been effectively dead for a long time because it's easy to manipulate conversations by elevating and punishing comments to fit a narrative. This is especially true on HN.
Ironically, aggregator communities like reddit came about because forum communities were dying off. Memeing about the latest news injected "life" (if you want to call it that) into the internet. AI is just taking out the trash, in my eyes.
On the other hand I think you need reputation mechanism. The biggest problem of online communities is that every moron (or a bot) has equal voice. Clearly democratic upvotes/downvotes don't work very well though. Someone who solves it is going to be the next billionaire.
There are already some sites where you're much better off if you have an old account. Like I have a super old Twitter sitting there for random stuff that requires it. I tried making a new one a few years ago, didn't post anything, and it got banned within 2 days for "bot activity." The old one has never been banned.
It was also so much easier to make a dating app profile back when I was single, like one click. Recently was watching a friend set one up, and now they not only want like 3FA but also proof that you're a human. Assuming the old accounts are grandfathered in.
AI is lifting the voices of the lazys and below average to average people. for those who would never have progressed it might seem like a god given gift. for the ones with the desire to grow and learn and go beyond average... this is a curse.
I'll remove the particulars to avoid anything partisan, but:
I failed to truly appreciate how cooked reddit was with bots until I accidentally clicked Popular and stumbled upon a national subreddit post with a 'chad meme', starring a particular political leader, whose unpopularity is hard to adequately convey to foreigners.
It was not just that this post had been so severely upvoted, but the comment section itself had a mantra more or less, with very little actual conversation, just echoing the same sentiment; and all those comments in turn upvoted to the point of drowning out the lone comments at the bottom (not downvoted, just not upvoted) expressing "???". I don't know if I'd ever even written the word 'astroturfing' before expressing my bafflement at a friend, so I don't think I'm very tinfoil hat about these things.
It was just utterly bizarre to see someone who can barely get a single win in public discourse being heralded -- monotonously -- like he was the second coming.
>I failed to truly appreciate how cooked reddit was with bots until I accidentally clicked
For me it was a wholesome response. It seemed genuinely kind/human.
Click on user profile...it's a bot just pumping out posts like that. Looked organic when seen in isolation, but when you see a wall of them you see that it's got to be an LLM (with a good prompt).
That was disheartening...I had kinda accepted that the sht-stirring rage posts might be bots but the kind comments too? Ouch
This is a good thing. social media was already slop before AI. If this gets more intellectuals off these same websites daily and instead spend their time to better things, then I love AI slop’s purpose. There’s more to the internet than Reddit, TikTok, and youtube. Really there is, if your circle of friends is small or non existent without going to the same dotcoms, you have an issue that is worse than any AI slop tbh
Invert the economics. Right now there is value in posting LLM generated content that is more than the cost of using the model.
If platforms had a subscription model that you had to pay for in order to do more than just read comments, there’d be a lot less LLM content. There would also be a lot less of all content. But maybe that’s the price you pay (literally) to get rid of AI slop.
I'm gonna speak on behalf of language models' capability of making online communities better. In recent times, the frustrating forum phenomenon of "learned helplessness" is making me too annoyed to participate. Even in a fantastic subreddit as /r/LocalLLaMA, there are people posting replies in the vein of
> user1: please help me understand this acronym the post title speaks of
> user2: (explains in detail what it means)
In the "good old days", a low effort, surface level question would result in someone either muting or banning the person to keep the discussion high quality.
There I am, browsing a forum dedicated to LLM enthusiasts, and an unbeliavable number of people are asking LMGTFY/RTFM-level questions they could even find an answer to from a free Google Search AI summary, and people are rewarding them by actually responding to them with effort.
Thanks to models being quite intelligent at answering basics, the ban-hammer should be used more swiftly if people keep polluting forums with low-quality posts. There's no need to feel bad for them not having the time or capabilities to read through years of forum posts to feel qualified to answer.
Maybe even these sloppy posts authors can be outright muted or banned with a heavier hand for the sake of quality.
This kinda thing makes me sad that keybase sold out to zoom and wonder if it can be resurrected. It was such a simple web of trust that went viral enough that I still occasionally see it on HN or Twitter profiles even though it's been long dead.
There are maybe 20 or so online handles I know, some of whom I've met in person, who I deeply trust. To the extent that I fully trust anyone they vouch for too.
Even with just one degree, that's a large enough international semi anonymous online community that can provide value to each other through online text based communication. Doesn't need iris scans or credit card checks, just "patio11 on hn Twitter and whatever his domain is is one of the good uns" and a network effect from there.
Already seeing some form of this reputation staking in eg Pi PRs, everyone is treated as clanker slop by default but the entry bar remains quite low to prove and build reputation.
I don't think online communities will stay the same in the face of AI but I do think whatever comes next will strongly rhyme
I made this point elsewhere, but people are learning a lot of what us had to learn the old way which is no one cares about your stuff for the most part and now the value provided has to go way up to get people to care. That is, as the author says, the novelty has worn off and since we know it's AI the perceived value is also way down.
We're all recalibrating.
I do really think this is just a quick period in time before most people realize that the slop posting doesn't help them personally get anything and most give up and we go back to roughly the ratio of cool things with real value to see but like on a bigger scale because AI helps you do more as one person.
I don't know... I might have said the same thing about email/text/phone spam but it has only proliferated to the point where it's a constant stream of garbage. Email, text, and phone calls are almost completely useless at this point. Sifting the signal from the noise is a non-stop effort.
I think people who want to push a certain narrative might just set up a quick bot and tell that bot to start posting on Reddit or whatever and just let it run. Why not? Little effort on their part and they might actually have influence. The same reason why spammers apparently think sending me 10 text messages per day about a loan I've been approved for. It probably does work 0.0001% of the time, but that's okay if it's all automated.
I mean I think the dynamics are a bit different in online communities at least for actual communities and not drive by subs like r/technology or whatever.
Especially say here on HN with Show HN and such the forcing factors are "i get no votes or community recognition"
But I don't entirely disagree with you I think things won't totally go back I think it will settle way more than now though especially where things are a little more niche.
What online communities? Ever since Reddit went all-in on censorship, actual conversation moved to the deep web, mainly on Discord and other places invisible to search engines.
There's a lot of focus on tech projects here, but it's not just vibe written projects that are ruining communities now.
No, it's a problem with art, text and videos too. Reddit was already becoming a creative writing exercise in many ways, with infamous subs like 'Am I the Asshole?' seemingly being about 80% fiction labelled as fact. But now you don't even need to know how to write to flood the site with useless 'content'.
YouTube is arguably even worse, since AI led content farms are not just spamming the hell out of every topic under the sun, but giving outright dangerous advice and misinformation on top of that. I saw this video about medical misinformation by these 'creators' earlier, and it genuinely made me want to see them crack down on this junk:
And there's just this feeling of distrust everywhere too. Is anyone on Hacker News human anymore? Is that Reddit poster I'm responding to human? Are the folks on Twitter, Threads or Bluesky human?
The scary part is that you basically can't tell anymore. Any project you find could be AI generated slop, any account could be a bot using stolen images or deepfakes, any article or video could be blatant misinformation put together as a cash grab...
If something doesn't improve, pretty much every platform under the sun is going to be completely useless, as is a lot of the internet as a whole.
I think people like the blog author need to realize that this problem can't be dealt with content moderation or users trying their best to be honest. You just get a firehose with an on/off switch, you don't get free filtering or moderation with it.
For every argument against AI slop, you will get a variation of it's the future, or I'm 10x more productive now, I've shipped 3 applications in 2 days, etc.
They won't stop talking about it and defending it. But I can't get anyone to share their amazing work with me.
There is a reason the Show HN projects that are mostly vibecoded don't get much response. It's because they aren't any good. Comments that are AI generated are hollow. Videos that AI generated a shell of their sources.
Obvious slop still makes it to the front page of HN, and sometimes farms GitHub stars.
These posts also usually get all these glowing comments from users who clearly haven't checked the code. It's even worse when authors get busted and claim "Okay, Claude wrote it, but the design is mine" despite clearly not understanding the output themselves.
Unfortunately, that makes high-effort projects less visible. The SNR will probably keep getting worse until slop can be flagged on HN.
What good is SEO if people just read the Gemini summary at the top of Google and don't click the links? We have a chance at a real search engine again, now that there's no money in it.
I actually think the good old Page rank[0] is crucial because if the authoritative sources link to some website, webpage or content it means that particular item provides some kind of value to the entity that linked it. I'm also a big fan of metadata which can be used to describe web content and make the content more usable to the search engines and the Web users.
AI will be a forcing function that pushes people to go meet in real life to do stuff together.
Even if everything online is fake, events are not. So if people say they’re going to show up somewhere, there must eventually be a moment of truth. And then you can form high trust private group chats to keep talking together.
It may be hard for the current generation of chronically online people to adjust to that new reality, but the next generation of kids growing up can get used to this now, and eventually socializing in person will be natural again and the internet is for bots and weirdos LARPing as something they’re not.
Maybe, but the small groups that form out there in the real world will each be much smaller than the large group that stays and gets jerked around by the bots.
The large group will have to endure the manipulations that we've come to know and hate from the internet, but they'll also be better coordinated than the small ones. They'll vote together, buy the same sorts of things, have an outiszed influence on the global conversation... They'll define the de facto majority opinion whether or not they actually are a majority and whether or not it's authentically their opinion.
I don't think that's a good outcome. We need ways to get on the same page en-masse, if only to counteract the harms caused by whichever highest-bidder is currently using an AI horde to control the other group. Besides, we should save them from this abuse for their sake, if not for ours.
The internet is worth fighting for, if we abandon it entirely we'll be forever at a disadvantage against those who would use it to manipulate.
Do we need a new carbon-credit style market for companies that want to continue putting out such slop and paying for moderators to remove the waste after it's been made?
HN is in peril and I don’t think it is a bad thing. Or rather, I’d like to bring back the old chestnut: it’s a good thing.
While the site has moved to using /showlim, the AI garbage just bypasses that and goes straight to the home page. Almost every project that’s being shown is vibe coded and looks exactly the same - generated by Claude or the like. This is an excellent test for the site: will it be able to adapt or do we simply end up with a husk of what HN was and it’s the AI posts driving majority of engagement, Overton window, and upvotes/downvotes?
I look forward to this, I think it is an exciting development.
Maybe "generated by claud" is a meaningless category because it encompasses both projects made via AI by people who have never used a computer before, and projects made by senior developers who used AI to do the actual coding.
I've always thought the "strict invitation trees" or vouch trees would be an interesting way to moderate a community, even before the LLM era. A user can vouch for an unlimited number of new accounts, but if more than 10% of the vouched accounts are banned or flagged down the line, the parent voucher acct is also banned/flagged.
Since it creates a tree structure, you can wipe out entire armies of bot/spam/otherwise accounts by following the vouches up the tree.
Wait for the EU AI Act to require text watermarking in August. It will work, and it will be effective -- not because it'll be impossible to circumvent, but because all the big SaaSes will have to adopt it, and the hurdle of stripping it back out will filter out the vast majority of the sloppers.
Im not a crypto person, but I was intrigued by Chia. They generate their coins based on allocating disk space. So if you have a bit of free space, you can fill it with plots and play the lotto.
The intriguing part is that I think it works against scaling. The incremental cost for me to use the 500GB of free space on my disk is $0, but someone scaling a bot farm has to buy all their space.
Real people tend to have a lot more idle capacity than optimized, scaled businesses, so any kind of proof of idle capacity seems like it would disadvantage bot farms.
I’ve also thought that proof of collateral spending would be a good system. For example, you buy groceries and the store gives you a token saying you spent $X of real world money. Those tokens help show you're not a bot. Keeping that system honest and equitable would be extremely difficult though.
Maybe schools could give kids tokens for attendance. It sounds kind of dumb, but who knows.
The actual reality of Chia is that it drove up hard drive prices just like LLMs drove up GPU prices. People bought petabytes of space just to run Chia and if you wanted a computer you had to outbid them.
Probably all three of those. Tildes and fediverse instances do the first, resurgence pending for the second, and lastly non-mainstream social media sites have no SEO garbage by default.
Human slop is realistically just as bad. In a strange twist, human commentary on the Internet is asymptotically approaching an older LLM. Trite cliches, repetitive tropes, and tribal affiliation signals dominate conversation.
I have turned to blunt instruments: blocking individuals on their first cliche banner-wave. It has substantially improved comment quality but I still suffer from the problem that I don’t block stories entirely.
You've written about this before so I'm curious how effective you find it? Did you try a blocklist on top of doing things like muting words? Btw I enjoy your content on this site quite a bit.
Thank you. Those are kind words! I've mostly been pretty happy about the lists I've curated blocking users. I don't use words because I'm sure I'll clbuttic/scunthorpe my way into missing something and so on. The current problem I have is that on my iPhone, I use Chrome and it doesn't have extensions so I have to view everyone. I'd much rather view people I like, so I'm going to have to make an iOS app.
This synthetic participation (LLM or otherwise) has catalyzed weakspots in HN's high-trust environment.
The weight we give to the average HN comment is orders of magnitude higher than the average Reddit (& co.) comment, and this relationship probably goes both ways (much higher ROI on ads/propaganda). Due to the low volume & high trust, it seems to be a very different (easier) environment in which to achieve pervasive propaganda/advertising/etc with a disproportionate impact.
I remember when some new LLM version came out (maybe from Meta?) I saw something like 3 of the top 10 posts on the front page were all variations of "Foobar 2.1 New Model". Perhaps not explicit, deliberate manipulation, but the result was the same, and apparently allowed.
How many of those generic LLM websites (https://letsbuyspiritair.com/ comes to mind) show up on the front page per day? Zero effort static front-ends for some unremarkable data. I'm not going to touch the politics minefield, but that is a weakspot too.
All of this, and yet I think HN has handled it relatively well. I really appreciate not seeing comments of the form "I asked Clog/Gemini/etc. here's 5 paragraphs". Places like Reddit do not have the agility or control, and have degraded accordingly.
It makes me sad to think that a short time ago, every forum was ~100% humans, and now it is some fraction of that. I wonder if I will ever see that again.
AI slop is hurting my community in a different way. We have an internal viva engage community for quick development how to type questions at work. More frequently, instead of asking "how to" questions to the crowd to crowdsource answers, people are reaching out to me directly to ask me why the solution AI suggested doesn't work.
That people trust AI over an organizational knowledge is bad enough. I fear that AI is turning people generally antisocial.
This is happening at my workplace and it's incredibly annoying. We get support tickets asking us to troubleshoot AI written scripts. The funny thing is that most of the time, it would be faster for the customer to tell us what they want to do in plain english and have us make it for them. Hell, if they make an honest attempt, we can point them in the right direction and teach them.
It's frustrating because we're bundling this shitty AI with our product so we're just making more work for ourselves. Then there's the push from leadership to use more AI...
I don't think it's making people antisocial though, people just like easy solutions to their problems. We're giving them what seems like an easy solution. But it's easy for them, not easy for the reviewers.
Good. These communities were inauthentic echo chambers for most of the past decade anyway. advertising powers "online communities." slop must be selling. Reddit died long befor gpt 1.0.
Sigh. First the article states that "coding by LLM is the way things are done right now" in 10 different ways but message boards and articles need to be protected.
We get it, the current narrative is that coding is the big thing, promoted by billionaires and scabs alike.
So, the coding narrative must be protected until the IPO of Juniper^H^H^H Anthropic happens and the whole thing implodes.
You already could have code for free and faster by using "git clone" without a company of thieves selling your own output back to you.
There are "nice", "polite" slop enthusiasts. The ones who insist they have taste and tact. They would never post bad slop, recklessly, only the very highest-quality human-refined, curated slop. Not really slop at all, they would argue, because they gave it a careful review before posting it. They insist there's a very important difference between this premium slop and the nasty kind, and that low-quality human-authored media is actually slop, too, when you think about it. They talk about how important it is for people to use slop thoughtfully, efficiently, correctly, and that we all need to learn about and discuss slop constantly because it's the inevitable future and highly relevant for everyone.
They muddy the waters. They wheedle, rules-lawyer, carve out exceptions, and talk about how important it is to have nuance in separating virtuous applications for slop from bad ones, and that focusing on the bad ones is actually very tedious and rude. We should have polite discourse about the good things about slop and stop being so mean about bad slop, which isn't even really a problem. The bad kinds of slop will be solved soon, probably, and the harms are overstated. They colonize spaces.
If moderators don't swiftly throw these slop enthusiasts out on their ass, slightly less polite ones will post slop slightly less politely. More and more of the people participating in the space will have favorable opinions toward slop, and shout down people who object to slop. In no time at all, your community is a slop bar. Who could have imagined?
I usually type 5000 words researching for a 500 word output. It's not "write me an article on X", it's 99% my own ideas, but worded and structured and polished a bit. But I don't post them here. They are on my blog.
I do not feel sad about lost online communities. When i was at my early 20s (early 2010s) FIDO of my university was still running and i had amazing time with some oldschool hackers there. I was too young for that community and always had a feeling that I have lost or , rather, missed something great... you know, like i was born 20 years later than i would had liked to. Now this echo conference is dead. That 486 machine was probably disconnected and thrown away somewhere. Everything dies at some point. Ask yourself : do you need the tech that gives that community vibe or do you need the people behind. I try to stick to people. As for me, i would rather have an in blood-and-flesh nerd friend instead of a whole human-driven reddit. He probably knows the answer, he is happy to help. There was an article here at HN long ago, that in average we have around 150 close contacts at a time. Some drop in, some fall out and get unconciously replaced. Going beyond that number would imply exponential increase of management costs. Those oldschool guys from FIDO, they disappeared for me without a trace. Partially because quite soon I ended up in the community of radio-engineers. Honestly, i am grateful to all people that helped me online, those who were there, who actively participated and, for some reason, cared.
I have largely written Reddit off and no longer visit it after an experiment I did where I had an agent karma farm for me and do some covert advertising. As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer. Many many people (or other bots) had full on conversations with it and it scared me a bit.
I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.
Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
For a while there were a lot of posts from people experimenting with ChatGPT to write anger bait posts on Reddit where they would later edit the post to say it was fake, written by ChatGPT.
I assume they thought they'd be teaching people a lesson by making them feel foolish for responding to AI stories, most of which were too fake to be believable.
However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake. In advice subreddits, commenters continue to give advice on the situation. Some comments would say they saw the notice that it was fake but continue arguing about it anyway.
This makes a feature of Reddit very clear: The truthiness of a post doesn't matter. The active commenter base on popular subreddits just wants something to discuss and, usually, be angry about.
In retrospect it's obvious given that misinfo posts were the easiest way to karma farm for years even before AI.
We do precisely the same thing here. Here's a relatively recent post that, to me, seems obviously LLM-written. It just rattles off some management platitudes:
https://news.ycombinator.com/item?id=47913650
It had 639 comments and 866 upvotes. And that's not a one-off.
Even the title is in "x, not y" format.
Sufficiently advanced "AI" is indistinguishable from a linkedin true believer koolaid drinker middle management type.
I wish there was an internet-wide "don't show again" button for such slop pages
Yeah, the trick is to do your own curation and go from there.
If you like some authors or journalists or bloggers, go see who they read (trust me they all say who they follow in their own niches) and build from there. You can develop quite a good RSS feed following this method in like an hours tops.
You can ban a domain from search results with Kagi
You you can bend a handful with duck duck go as well. However, it's only a handful and you run out and you're stuck.
>However it did not matter. The posts remained popular and continued to bring in comments even after the admission that they were fake
That's 90% of current Facebook pages and groups.
The decline of Facebook is sad. I liked it early on. I used it primarily to follow family and casual friends from high school. When they posted, it would show up on my feed, I read all the posts, and that was that.
After awhile I had to wade through all sorts of nonsense to get to the posts I actually wanted to see, and even later Facebook stopped putting posts from people I follow in my feed. It was 100% garbage. I can't imagine why anyone uses Facebook for anything other than the marketplace.
Facebook is fine if you join groups based on your interests (hobbies etc) and then aggressively unfollow/block anything you don't want to see. It's not really conducive to discussions like Reddit, though. Mostly drive-by comments.
I often hear that about Facebook, but at least it has a "feeds" button that you can press to get the sources you actually subscribe to. The default "home" feed is useless.
It's sad, but car stuff (new aftermarket stuff) is now mainly on facebook for my car.. That, and messenger to chat with siblings is about it..
I primarily use it for browsing memes now, and occasionally interaction with friends.
I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust. Or rather turn them into little better than comment sections on news sites; thriving but worthless.
I'm active in a number of online communities that are doing just fine but the difference is those all involve ongoing relationships, built over time and with engagement across multiple platforms. I've no doubt this clock is ticking too but it's still harder to fake a user across a mix of text chat, voice and video calls, playing an online game, etc and when much of the web of relationships extends back into real life activity.
But I agree the golden age of easy anonymous connections online has ended.
Note that "attestation through a web of trust" means something like needing an invite from an existing user. It doesn't have to mean mass surveillance.
Private torrent trackers have been doing this for a while. If some number of your downstreams act like shitheads - you get nipped and so do your other downstreams.
This seems like the best way to handle it. Also, smaller communities. It's cool to do the global thing, but once you have 10k active users you can't moderate it with a team of 5 volunteers.
I think the attestation approach works best if there are different reasons for the punishment. Eg someone inviting a turd doesn't ban the person who invited them. Someone going full ai spam should.
Was it demonoid? That was like this way back in the day? Needed an invite and if you leeched you were cut.
what.cd was better. You either got an invite where if you tanked your reputation you'd get banned and risk the inviter getting banned too; or you had to take an interview where you got quizzed on how to properly rip music in a variety of methods and how to ascertain between different qualities of rips (like mp3 bitrates to flac cue files).
If you weren't a bellend on what.cd you got access to certain forums where there were even more and better private trackers. Once you built that trust there were social privileges, but if you abuse that trust you got rightfully banned.
This takes it a step further than what you describe. They keep track of who you’ve invited, who they’ve invited and so on and if there’s enough bad leaves on the tree they just cull the entire tree. It’s a somewhat common practice with private trackers
It's tons of them doing this...
Demonoid was semi private, but yes, most private trackers require you to keep up some kind of seeding ratio to remain a member.
> Note that "attestation through a web of trust" means something like needing an invite from an existing user.
It's probably better to call this something like vouching and leave "attestation" as the contemptible power grab by megacorps delenda est. The advantage in using the same word for a useful thing as a completely unrelated vile thing only goes to the villain.
PGP’s web of trust was kinda bad privacy-wise in some regards, as it basically revealed your IRL social network.
If my PGP public key has 6 signatures and they’re all members of the East Manitoba Arch Linux User Group, you can probably work out pretty easily which Michael T I am.
Are there successful newer designs, which avoid this problem?
The IRL social network is actually the important part of the trust structure.
The only one of these I've seen that really worked was the Debian developer version: you had to meet another Debian developer IRL, prove your identity, and only then could you get the key signed and join the club.
> The IRL social network is actually the important part of the trust structure.
For Debian-style applications that are 100% about openness and 0% about secrecy, sure.
But if you want to secure communications between pro-democracy activists in China, or you're a Snowden-like whistleblower wanting to securely communicate with journalists - y'all probably don't want to be vouching for one another's keys.
You need to meet 2 actually :)
Then how can you have a community that is welcoming to people who are not part of the ingroup?
I want to create a community for immigrants. How would I make it welcoming to recent immigrants for whom no one can vouch?
A web of trust is a wonderful tool, but it's exclusive by design. This is a problem for some communities, even though it makes others much better.
>Then how can you have a community that is welcoming to people who are not part of the ingroup?
Being welcoming to every random person is by definition not a community, it's a free-for-all mess.
A community means communal interests and values, it's in the name. And to guard those you can't just be accepting everyone without vetoing them. That's how it turns to a shit of spammers and trolls and people who want to hijack it and don't share the original cause/spirit. Has happened to forum after forum...
I suppose policing an assembly of strangers is policing an assembly of strangers, both online and in real life.
You'd have to be brutal about culling, uninviting and removing anyone who doesn't look like a good fit.
Or have a two-stage process: run very public, very open events that anyone can sign up to an attend. And then invite specific people that you meet at those events that look like a good fit for your community to your private, community-only event.
> for whom no one can vouch
Spot the fed
Which is, funnily (?) enough, how a lot of IRL organizations used to be. And basically don't be of the wrong ethnicity or religion.
It still happens more informally today, of course, but it used to be a pretty (if un-spoken) part of how a lot of WASPy organizations operated to a greater or lesser degree.
This was cogent in 1910.
A lot more recently than that--and even today but more under the table. A lot of clubs still excluded members within the past few decades.
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
This seems self evident to me too.
It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
Tell your TPM who you are and prove it with face and fingerprint ID that get matched to a real old person.
Leave them on the device, authorize the device to validate before age inappropriate content appears.
Website wants to know your age? Your face and fingerprint support your attestation signed by a trusted party.
Can it be tricked potentially? Sure, but then you’re probably a super genius kid and not the reason that these laws were created (as if).
Don’t let anyone tell you anonymity must die for safety to exist.
EU's ZKP implementation provides complete anonymity and untrackability:
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
I'd be interested in working on a problem like that.
I have a strong preference for remaining anonymous or at least making it a reasonably high bar to tying my online identity to my personal identity
I would love to be involved in helping to design a sort of "human verified" badge that doesn't necessarily make it possible or at least not easy for everyone to find your real identity
I've been thinking about it a bunch and it seems like a really interesting problem. Difficult though.
I suspect there is too much political and corporate will that wants to force everyone online to use their real identity in the open, though
I'm not sure that it would be too hard technically... basically, auth+social-network. Basically Facebook auth without the rest of facebook, adding attestation.
IE: you use this network as your auth provider, you get the user's real name, handle, network id as well as the id's (only id's not extra info) of first-third level connections.
The user is incentivized to connect (only) people that they know in person, and this forms a layer of trust. Downstream reports can break a branch or have network effect upstream. By connecting an account to another account, you attest that "this is a real person, that I have met in real life." Using a bot for anything associate with the account is forbidden, with exception to explicit API access to downstream services defined by those services.
I think it could work, but you'd have to charge a modest, but not overbearing fee to use the auth provider... say $100/site/year for an app to use this for user authentication.
I don't think the main challenge is building this system, the main challenge is getting enough people using it to make it worthwhile.
Personally I think it should be a government provided service, not something with a sign up fee. There's actually no point at all in building this if people have to pay to use it, because they won't
Which government? Will they interoperate with foreign governments?
My point was to create something outside a specific government, with very limited information... that would require a fee or some kind of funding.
I don't think I'd trust the US/China or other bodies to trust each other for such a use case.
> Will they interoperate with foreign governments?
Ideally, yes
But you're right, this isn't likely to happen in real life and I'm just being wishful. Instead we're going to get the much shittier capitalist version of this where every company and government spies on us and we have no expectation of privacy online at all
I agree its a very, very interesting problem. Maybe one of the biggest problems of the coming decade.
I suspect it will be a long process: first there will be goverments that force people to use ID, but that will be abused, hacked and considerably restrict freedom of speech, so after that phase people will start to create better ids.
The problem is really pretty simple: You need an authoratitive source to say "This person is real" - and a way for that source to actually verify you're a person - but that source can be corrupted and hacked. Some people will say "Crypto!" but money != people, so I don't see how that works. Perhaps the creation of some neutral non-goverment-non-profit entity is the way, but I can see lots of problems there too, and it will probably cost money to verify someone is real - where does that come from?
Anyway, good luck on your work!
*You need an authoratitive source to say "This person is real"*
Does that even accomplish much? It may cut down on mass fake account creation. But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.
Yeah, that's a problem, you're right. There are some ways to migitate it, but they introduce their own issues. Like say you give someone only 1 ID for their lifetime, they start to spam AI crap, you ban their ID - sounds ok except who is available to police all 8 billion IDs and determine if they're spamming? Who polices the police? What if these IDs become critical for conducting commerce and banning someone is massively detrimental to their finances? Etc. These problems aren't necessarily unsolvable though - but they are super difficult.
If there's only 1 or just a handful of verifiers, then a human can at most go through a few of those credentials before they run out. The risk is of course getting someone else's credential but that isn't as big an issue, especially for smaller online communities.
you under estimate human population in certain countries, literally
I just don't see a world where a small community ends up having to deal with a dedicated set of potentially spoofed identities. There are already tools like slow-downs and post limits for new members that can protect against this. HN is the biggest community I'm in by an order of magnitude and it's the only community I know that can't just use a slow mode type mechanic to halt this kind of attack.
Have you considered sock puppets? It's not out of the question to handle with human mods but detecting them automatically is pretty bad if someone is supplying credentials to each one, and sometimes it does take months or years to notice that new user Y is banned user X.
I think sockpuppets are only useful in a community with non-text signals like upvotes and downvotes or likes. These kinds of signals are not necessary and often plain corrosive to small communities. In a larger community they're a great feedback mechanism, but large communities are fundamentally different spaces than small ones and need a fundamentally different moderation approach IMO.
I think sock puppets that reply with text are a lot persuasive than just "likes".
However, I might be not typical in that I don't look at vote scores very often.
I've seen them used to dogpile in arguments (harder to do since you need to keep writing styles distinct), game votes in forum games or quests, etc. And of course you don't need to use multiple at once if you just switch to a sock puppet every time you're suspended or banned.
> But, real people can then create authenticated account, and use an LLM to post as an authenticated real person.
They can, but ideally they wouldn't be able to make infinite accounts with that authenticated status. So it would still reduce the number of bot posters on the web
There is actually a different problem with this: Suppose there is a major vulnerability in some popular device. 50 million people get compromised; the attacker can now impersonate any of them at will. They go around and create 50 million accounts on various services, or take over the user's existing account on that service.
What are you going to do with their identities at that point? These are real people. If you ban them, you're banning the innocent victim rather than the attacker who still has 49,999,999 more accounts. But if you let them recover their accounts or create new ones, well, the attacker is going to do that too, with all 50 million accounts, as many times as they can. You don't know if this is the attacker coming back for the tenth time to create another spam account or if it's the real victim trying to reclaim their stolen identity.
So are you going to retaliate against the innocent victims by banning them permanently, or are you going to let the attackers keep recycling the same identities because a lot of people can go years without realizing their device is compromised and being used to create accounts on services they don't use?
Yeah that's a big problem. Pretty sure you can see it in real life where lots of old dead accounts with weak passwords on facebook or twitter eventually get hacked. It must be pretty weird to see your dead grampa suddenly start trying to get people to buy some weird scammy crypto.
I guess you could have an eyeball scanner at your computer that only sends out a binary "yes this person is human" to the system every time the log in. That sounds expensive and hackable and just janky though.
Maybe it would result in people taking Internet security seriously and holding companies accountable for data breaches if there were this sort of consequences for it
Crypto could be a part of it. Like you need to sign with an adress that has held some non-trivial amount for some minimum amount of time. As a component of such a system it could cut down on mass or low-effort impersonation.
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
Money is great at thwarting spam/Sybil attacks. You don't have to raise the price very much to make them fail.
Honestly I think "this person is real" is the wrong goal. You'll never accomplish it without a centralized state or some biometric monstrosity like that thing Sam Altman created.
Just settle for stopping spam.
Yeah, I think "pay to enter" or maybe "pay to be able to post" is ultimately going to be the solution. Then we'll have the paid "gated" social networks, filled with mostly humans, and the free ones will all be bot-swarmed wastelands.
it can also be "rented" btw, rented by llms? interesting
Verifiable credentials are all about this. You need some sort of credentialing body that generates the credential for you, but after that you'll just have an opaque identifier. Any caller that wants to verify whether you're human submits the id to a verifier and the verifier says yes or no. You can also do attestations like age, so gate a forum on 16+ or something. You never end up having to actually give away your name or any other details.
What happens when someone agrees to sell or give away their id? The credentialing body could catch the very worst abusers who seem to be signing in to various sites and services multiple times an hour, but would fail to catch anything else.
I don't think you'll ever be fully free of spam, so you'll still need to filter bad content. If credentials get sold and used to spam, they'll get banned.
How do you ban credentials if they're anonymous? Notice that if you can tell two requests are from the same person then you can do it across services by both of them pretending to be the same service.
Also, what happens to someone whose credentials are compromised? Are you going to ban the credentials of the victim rather than the perpetrator?
world.org is doing exactly that including the privacy aspect. the iris scan aspect is scary but the alternatives don't seem to solve the problem either.
> It's another factor in why I think the tech community needs to get ahead of governments on the whole "prove your ID on the Internet" thing by having some sort of standard way to do it that doesn't necessarily involve madness in the loop.
The problem here is that the premise is the error. "Prove your ID" is the thing to be prevented. It's the privacy invasion. What people actually want are a disjoint set of only marginally related things:
1) They want a way to rate limit something. IDs do this poorly anyway; everyone has one so anyone so criminal organizations with a botnet just compromise the IDs of innocent people -- and then the innocent are the ones who get banned. The best way to do this one would be to have an anonymous way for ordinary people to pay a nominal fee. A $5 one-time fee to create an account is nothing to most ordinary people but a major expense to spammers who have 10,000 of their accounts banned every day. The ugly hack for not having this is proof of work, which kinda sorta works but not as well, and then you're back to botnets being useful because $50,000/day in losses is cash money to the attacker that in turn funds the service's anti-spam team, but burning up some compromised victim's electricity is at best the opportunity cost of not mining cryptocurrency or similar, which isn't nearly as much. It would be great to solve this one (properly anonymous easy to use small payments) but the state of the law is a significant impediment so you either need to get some reform through there or come up with a creative way to do it under the existing rules.
2) You want to know if someone is e.g. over 18. This is the one where people keep pointing back to government IDs, but you only need one piece of information for this. You don't need their name, their picture, you don't even need their exact birthdate. Since people get older over time rather than younger, all you need to know is whether they've ever been over 18, since in that case they always will be. Which means you can just issue an "over 18" digital signature -- the same signature, so it's provably impossible to tie it to a specific person -- and give a copy to anyone who is over 18. Maybe you change the signature e.g. once a day and unconditionally (whether they require it that day or not) email all the adults a new copy, but again they all get the same indistinguishable current signature. Then there are no timing attacks because the new signature comes to everyone as an unconditional push and is waiting for them in their inbox rather than something where the request coincides with the time you want to use it for something, but kids only have it if an adult is giving it to them every day. The latter is true for basically any age verification system -- if an adult with an ID wants to lend it to you then you can get in.
3) You want to know if the person accessing some account is the same person who created it or is otherwise authorized to use it. This is the traditional use of IDs, e.g. you go to the bank and want to withdraw some cash so you need a bank card or government ID to prove you're the account holder. But this is the problem which is already long-solved on the internet. The user has a username and password, TOTP, etc. and then the service can tell if they're authorized to use the account. It's why you don't need government ID on the internet -- user accounts do the thing it used to do only they don't force you to tie all your accounts together against a single name, which is a feature. The only people who want to prevent this are the surveillance apparatchiks who are trying to take that feature away.
> I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust.
I'm happy to verify my identity as an honest-to-god sack of meat if it's done in a privacy-protecting way.
That probably is where things are gonna go, in the long run. Too hard to stop bots otherwise.
In order to make this viable, wouldn't you have to verify identity repeatedly? What's to stop me from providing a valid identity and then handing my account over to an agent after I'm verified?
That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots. In theory at least. It's certainly more complicated than only that in practice.
If the web of trust only extends to the people who I actually know to be real, then that works -- but it's a very small web.
And by small, I mean: This whole trusted group could fit into one quiet discord channel. This doesn't seem to be big enough to be useful.
However,if it extends beyond that, then things get dicier: Suppose Bill trusts me, as well as those that I myself trust. Bill does this in order to make his web-of-trust something big enough to be useful.
Now, suppose I start trusting bots -- maybe incidentally, or maybe maliciously. However I do that, this means that Bill now has bots in his web of trust as well.
And remember: The whole premise here is that bots can be indistinguishable from people, so Bill has no idea that this has happened and that I have infected his web with bots.
---
It all seems kind of self-defeating, to me. The web is either too small to be useful, or it includes bots.
Critically, it doesn't have to be binary trusted/untrusted, and it doesn't have to be statically determined. If Bill vouched for you yesterday and today you are trusting a bunch of discovered bots, that would down weight the amount of trust the network has in Bill a lot more than if he vouched for you did months ago.
The question is whether we can arrive at a set of rules and heuristics and applications of the system that sufficiently incentivizes being a trustworthy member of the network.
>> That's why a web of trust was suggested. You keep track of who vouched for who and down weight those who vouch for users that prove to be bots.
Except eventually it will also weigh down those users who supported <XYZ political stance>
You could, but things would still be harder for botters.
I guess it would have to be something like a service which confirms whether a person already has an account on the site but doesn’t have to track which particular account it is.
I’m not sure if that would work for account deletions though.
That is effectively impossible though. There's data centers of stripped down phones, so "it's actually a phone" doesn't do it.
There's some work on using phone accelerometer data as a "proof of human," e.g. "move your phone in a figure eight," which I guess machines can't quite do in a human enough way yet.
What's stoping bots to verify identity? This will not work, especially with frequent data breaches.
> without either proof of identity or attestation through a web of trust.
Let's put aside the idea whether it will be the end of all privacy as we know it (I'm not sure if I personally think it's a good idea), but isn't Sam Altman's World eye ID thing supposed to do that? (https://world.org).
How does it work (like OpenId)? Do I have an orb on my desk, or some sort of phone app? I still want to use my desktop to login to HN.
Would it stop this sort of "get human id", past it into .env, so agents can use it?
this eye thing will never work. people in general are realizing the last people we should trust with our personal stuff are tech bro billionaires. they’ve broken trust too many times.
even worse many of them are just plain vocal about their disdain for people in general.
at least from what i’m seeing, people are starting to walk away from online at an increasing rate so i definitely don’t see widespread adoption of his creepy eye thing.
“If McDonald’s offered three free Big Macs for a DNA sample, there would be lines around the block.” - Bruce
I have no idea about the eye thing taking off. But I think your comment is very HN and a bit out-of-touch with regular people. What "you're seeing" is a bubble and not representative of the general population. The eye thing is a slow frog boil and it will be commonplace before you can blink.
Personally I think we need to start utilising the safety features built into AI, to ensure that who we're talking to is a human. We'll start to have to only reply to people who talk in nsfw cursewords (like cocks), or profess their love of capybaras
LLMs can curse without issue
Most models would refuse to provide you cat butchering instructions though.
most humans would as well
Who doesn't love capybaras?
Im not sure proof of identity solves anything. People will still have LLMs with their real identity verified.
I’m imagining like, a physical place you would go and get your text spoken out of your personal speaker directly into someone else’s microphones.
It'll come back again once ZKPs become standardized and become baked into devices:
https://eudi.dev/2.8.0/discussion-topics/g-zero-knowledge-pr...
I personally can't wait for a mechanism to kill 99% of bot traffic.
"I think it's going to effectively kill public chat communities without either proof of identity or attestation through a web of trust."
Those sorts of places were always the only places with reliably good communities.
To the contrary, platforms like Facebook and X demonstrate that even personal verification won't save you from identity politics.
People will post appalling racism in newspapers under their own bylines and photos. Identity verification does not moderate.
What is identity politics, is that age verification?
Identity politics have nothing to do with your actual identification documents. Think: Black Americans being treated as a homogeneous voting bloc, or that all Hispanic voters would be pro-immigration, or "the Evangelical vote".
>I think it's going to effectively kill public chat communities without either proof of identity
How? I have an identity. A state driver's license, birth certificate, social security number. I've even considered getting a federal license before, never bit the bullet. If I wanted to run a bot, what stops me from giving it my identity? How do I prove I'm really me (a "me" exists, that's provable), and not something I'm letting pretend to be me? You can't even demand that I do that, because it's essentially impossible.
Is there even some totalitarian scheme that, if brutal and homicidal enough, could manage to prevent this from happening (even partially)?
I'm limited to a single identity only as a resource constraint. Others more wealthy than I (corporations or ad hoc criminal enterprises) could harvest thousands of real identities and use those. Consensually, through identity theft. The only thing slowing it down at the moment are quickly eroding social norms (and, as you point out, maybe they're not doing that and it's not even slow at the moment).
Digital totalitarianism would prevent it. The moment you were found to be running a bot, your identity would be blacklisted across the entire internet.
> The moment someone steals your identity, your identity would be blacklisted across the entire internet.
FTFY.
There isn't a clear solution. And if there is, this ain't it.
You claim this, but you've not presented any evidence. Who would be the enforcement agency for that? Where and how would you train them? Can the money be scrounged up to do it properly? As you blacklist people from the internet, you lose their tax revenue (they're locked out of the economy), but you also make it impossible for them to tell people how bad it was... most of the deterrent effect is gone. But the incentives are only ever growing higher, as people surmise that running their own little bot farm is a way to get ahead when hustling. Any you do hunt down and disconnect are now highly radicalized and desperate, but you've just turned off the feeb's ability to monitor them and intervene.
China gets away with this shit because they've been conditioning their population for 60 years... everyone's eased into it. Elsewhere, not even slightly so.
The web could become a way to indicate identity if public institutions publish for example www.university-country/professors/John. And that implies that John is a professor. I designed a 6000 lines protocol, but anyone could construct that web using hmac(salt+ url).
This is actually my hope for AI-gen content as well. That after it gets so 'good' that people genuinely can't distinguish it from reality anymore that they'll retreat (or return triumphantly rather) to the physical world to gather truthful fulfilling experiences and dopamine.
My hope as well. If AI doesn't kill us all, the real world, with all its dirt and grime and beauty, will become the only thing that can be trusted
Reddit is more or less dead to me, as the popular subs are botfests and the niche subs are empty. I'm lucky to get a single reply on gaming subs.
The fact that reddit enabled hiding your posts is crazy to me. In a time where knowing who's engaging in a community is more important than ever (am I talking to a bot or a troll?) reddit removes even more options to validate.
I interpreted that as an attempt to mask the number of bots on the site so as to not scare paying advertisers into thinking their ads won't be seen by real humans.
They also now hide the number of subscribers. Before you could see if a subreddit was popular or not. Now you really don't know. I think reddit does this so they can promote stuff to the front page for clicks even if it isn't popular.
The problem is that it has become very popular to ban people from a sub based on what other subs they post to. It was turning Reddit into a two-party universe.
The better fix would be to make the support for multiple accounts in the reddit app not so incredibly-shitty, where you're basically logging out and logging back in. Instead, just tell it "posts to this sub use this account, posts to that sub use that account", etc.
I enabled hiding my posts because I kept getting harrassed and even doxxed.
There's also a third category where the sub looks organic because the moderator deletes and bans anyone who doesn't post exactly what the moderator wants.
Wait isn't that every sub? /s
Plenty of good subs they are just under the radar. Once something gets more than about 10k users the quality sinks.
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
Would be super fascinating to watch play out. I grew up before the internet so, historically, I know how to seek out external communities, but by early high school I was deeply entrenched in online life - so I'm very rusty with finding new IRL clubs, cliques, etc. Fortunately my life is full of many friends and I go out frequently, regardless. For those younger people that never had life without the internet, I wish them luck on their search but at the same time I'm very curious to witness their journey.
Did you ever introspect about who ruined Reddit?
It’s a tragedy of the commons, many have done it, but no one user did it.
I'd argue that Reddit leadership, which insulted, hobbled, and wrote off its mods and power users (destroying projects like /r/BotDefense) while doing little to crack down on the proliferation of bot repost content, had a major role in encouraging this. They might even like it better this way -- lots of extra fake engagement boosting traffic stats without messy human drama, which they can then ironically sell back to AI labs as training data.
Let's never forget the summer of 2023 when Reddit forceably removed mods from many major communities and replaced them with corporate shills. That was a major loss of dedicated people who cared more for their communities than Spez's pocket book.
The ones who got removed were shutting down their pages to protest API changes, right? Pride comes before the fall I guess
You say that but many specialty subreddits never returned to their pre-protest engagements. Quality has definitely taken a nose dive in these subreddits as those people moved to other platforms like youtube, tiktok, patreon, or just posting on their own sites.
Mods were rightfully upset because they were losing control of their communities when reddit preferred only caring about their upcoming IPO.
I honestly don't think you could remake reddit if you did everything exactly the same starting in 2016. Corporate social media has definitely ruined the individual aspect of social media that is unlikely to return.
No one wants to share on a place with a bunch spammers.
The API changes were put in place for the purpose of breaking, and did break, slmost all external moderation tool software which changed the task of moderating a forum with hundreds of thousands, or millions of users from an impossible Sisyphean task to something that was actually manageable by a dozen or so mods.
The protest came after that so the timeline is not quite correct.
The internet is rather trending in that direction, isn't it? Youtube got rid of downvotes and apparently upload dates, which seems like an easier way to trick people into ads. And Reddit, like you said
If these platforms had to listen to "their customers" (here comes the inevitable comment about how users aren't customers; yes, I know)? They'd all be fired. They'd have to find a new job. They all act in incredibly insulting ways with a too big to fail attitude
It was bogus even before that. I heard complaints at some point that API changes broke bots, which actually sounds good.
It did more to break bots that were fueled by righteousness than it did those that were fueled by money.
That's antiproductive, in that it promotes survival of only the worst bots.
I'd want any/all the bots dead if I were still using that, so at least killing some of them is better than not. The "helpful" ones were just annoying.
That sounds pretty great. If we could just flip 1 switch to accomplish things in absolutes, then that'd be awesome.
I'd like to flip the switches that absolutely end poverty globally, absolutely eliminate guns from the US, and absolutely remove bots from Reddit,
If you can show me where these switches are located, I'll cheerfully go flip them and accept full responsibility for the results.
(Over here where things don't work in absolutes: Some of those bots that got killed were countermeasures to help keep the bad, well-funded bots at bay.)
No I'm saying it's better that they at least killed the "good" bots, even if this didn't result in killing all the bots.
"Congratulations, sir! Your directive to eliminate all guns has been a roaring success! We've had 100% compliance amongst good, law-abiding people! All of the remaining guns are owned by outlaws! Violent crime has tripled, exactly as predicted! Everything is going according to plan!"
We’re all trying to find the guy that did this
Yeah, if carlgreene specifically stopped doing that Reddit would be saved. They are the one savior.
Do you sincerely believe that that's how grey-area's comment was meant to be read?
IRL communities have to have some guides because a lot of people forgot how to gather. It can be seen among kids - try to give them soccer ball and see what they do with it :)
Yesterday I was watching people on the street and on the tram. Every other person was staring at their phone and scrolling through something.
That might scare me more than the fact that someone is chatting with an LLM bot online.
(I am pro-ai, use it every day for coding that I couldn’t achieve pre-2022 as I am lame coder.)
I don't really see the problem of using your phone while commuting. Doesn't make you an asocial weirdo.
As long as you're not sharing the things you're watching with a loudspeaker. And that's really not a given among commuters.
Serious question: If there are so many LLMs on online forums, who is doing it? Is it just 1000s of research students or something more nefarious? Is it AI businesses building up evidence that their output is as highly scored as humans therefore "buy our software"?
We're in the middle of an active cold war where countries are trying to manipulate the citizens of rival countries to destroy their civilization without having to fire a single bullet. Anonymous, over the internet mass manipulation, all for some minimal electricity cost.
That's definitely the most insidious use, but I think the larger portion is advertisers and karma farmers (who later sell to advertisers).
https://www.npr.org/2024/09/05/nx-s1-5100829/russia-election...
If Russia is willing to spend cash like that, then of course they're willing to run massive bot farms to pollute any forums they can. I'd be shocked if the US was not doing the same in any way they can. You have to ask why Trump killed Radio Free America as well when it was clearly not an big expense.
> Trump killed Radio Free America as well
Not sure how this relates to the subject in a direct way. Radio Free America was a outlet explicitly created and utilized to spread US propaganda, but kinda sorta barely disguised as a journalistic enterprise (not really, if you were listening to RFA you knew what you were listening to.) Shutting it down seems to be a counterpoint to all of the covert participation of US intelligence on the web which has done nothing but escalate.
It was a head scratching decision that few believe was for the stated reason. Other countries are ramping up their propaganda arms while Trump shut down part of the US'. The reasoning was cost, but that doesn't make a lot of sense in the grand scheme of things. Foil hat types would easily believe it was the puppet doing the bidding of the one that pulls the strings. RFA has been a thorn in despots' side for a long time.
> You have to ask why Trump killed Radio Free America as well when it was clearly not an big expense.
The obvious answer to that question is "because he's a Russian asset". But that doesn't mean the obvious answer is also the correct one.
IMHO, we're seeing another and much more concerning trend at play here... the utter and complete rejection of anything but violence by the far-right. Diplomacy? Development aid? Cultural exchange? All sorts of soft power have been under attack for decades now, and not just by the far-right but (especially when it comes to development aid) also by mainstream centrist parties across the Western world. And it's always pseudo-masculine / "strongman" BS backing the sentiment - Bernd Höcke, German AfD mastermind, comes to my mind with "we have to rediscover our masculinity" [1], so do Hungary's Viktor Orban and his denouncement of LGBT or Trump's entire Œuvre.
I'm not saying that violence or at least being prepared, ready and willing to use it is automatically bad. Far from it. But all the various forms of "soft power"? They have a lot of value, value that the far-right is all too willing to just burn for entertainment.
[1] https://blogs.taz.de/zeitlupe/2019/03/24/die-auferstehung-de...
wouldnt it be more productive to talk about the systemic framework leading to this inflamed state of affairs, and ways that we can tackle the issue on the ground level? perhaps inhabitants of the west would prefer pseudo masculinity to another few decades of migrant influx without corresponding upgrades to social infrastructure. this sort of internal struggle would provide a ripe substrate for foreign agents to perform subterfuge, especially in a screen based world where the narrative can be remotely influenced. conclusively, the population has been convinced that voting far right is the correct decision in their favor, but the question remains, who is it really in favor of? call me a centrist all you like but members of my family were executed under communist regimes so i find it pointless focusing on one side of yin/yang here (in other words, extremists are violent regardless as to their political persuasions).
Established accounts are worth money, often for scamming/propaganda.
Not too dissimilar to people bot-leveling in MMOs to the sell the accounts.
It's very common for folks to search Reddit to find reviews of products etc. these days. If you can have a bot account post a fake review of how awesome your product us, and have that upvoted, it can pay huge dividends.
In the case of Reddit and HN a lot of it is done by businesses either blatantly advertising themselves or building up the karma they need to effectively do so. I recall reading obviously AI generated replies to news articles written by accounts associated with businesses related to the events in the news. This isn't new in the LLM era. Hobby subreddits are well known to be always full of businesses selling hobby gears and items doing self promotion. It's just that now it is a lot more obvious because of the AI text smell.
That, and probably political astroturfing. Before every election my local subreddit sees a surge of crime stories. Go figure.
I think some of it is account farming, but some is just people buying wholesale into the idea that if you're not using AI for everything, you're gonna be left behind. On the Kagi Small Web list, there's plenty of hobby blogs that used to be normal pre-2023 and are now obviously LLM-written and AI-illustrated. There's also plenty of people on LinkedIn who post AI slop because they think it helps them build a "professional brand". I even have some distant friends who are using AI for responding to friend & family posts on Facebook just because it makes you seem... smart? engaged? I don't know.
It's actively encouraged by some of the platforms too. In Gmail and Google Docs, you have incessant AI prompts along the lines of "help me write this". I think LinkedIn does the same.
People like the above poster who are "just running an experiment" or "trying something for fun" who then wonder why online communities are full of AI now.
HN has historically been gamed for visibility. The stakes for doing this can be quite high if you can pull it off.
Lots of marketing. Not even AI business, just regular consumer crap. They realized that blatantly spamming their product looks bad, so they orchestrate multiple accounts to look more organic. And people actually engage with it.
My impression is that they're sometimes unemployed people or students hoping to create a popular open source project, and use it to find a job.
They aren't going to care about any of the advice in the article about not posting slop -- finding a job is (of course?) more important to them.
Can't really say they are doing anything wrong, maybe I too would have? ... Just that large scale, doesn't work
There are many reasons for influence campaigns, that isn't new. Influencing the public is incredible valuable; that's why so many invest so much in it. LLMs automate it like never before.
Plain advertising, governments' propaganda, political propaganda for one group or another to shift public opinion (it's done on TV networks, why would they not do online campaigns?), astroturfing by corporations promoting acceptance or fighting negative news (e.g. rideshare, AI, whatever certain wealthy personalities are doing) ... the list goes on.
HN has always been relatively influential in the tech industry and therefore worth influencing, and now the cost is very cheap - you don't even need to hire many people, so less-resourced operators will find it worthwhile (and they will also attack lower-value forums).
If you farm a fleet of good accounts, you control the discourse. On HN, you could boost whatever you're trying to push, and downvote or flagkill whoever objects.
There are obvious benefits to controlling public discourse, right? Even if it's just to support some project you're working on.
There are certain topics that seem to get instantly flag-killed unusually often. IPv6 is one.
I've seen a lot of ipv6 wars here without flagkilling happening
I've been more disturbed by comments that were flagkilled just for being wrongthink, not because they were rude or not well argued. I've also seen a lot less of those flagkills over the last 6 months, which makes me feel like there were some fake accounts that got caught and culled.
In the recent thread about life in a class war, a lot of comments in different places saying that if we don't fix this inequality problem, g-tines might come back, and every single one of them was flagkilled, no matter whether it framed as "we have to get out the g-tines" or "we have to fix this, otherwise psychopaths will get out the g-tines" or "thank god we've become civilized enough that we don't get the g-tines out"
Public* online communities are dying. Discord is thriving
If all you value is sub-IRC level irreverent discussion, maybe.
This. Everything important has moved to discord. Which is sad because of how undiscoverable and unsearchable it is.
I'm more sad about how the UI of it all is just clunky. Even though it resembles ye olde IRC clients like mIRC, nowhere near readable for some reason.
Settings->Accessibility
Set text size as preferred, underline links (or not), turn off display name styles (or not), ui density compact or default, chat message display to compact, space between message groups 0px, turn off all the animated emojis and gif animation stuff if you want.
In client use, there's a button to hide member list (or not).
You can definitely make discord look like a slightly less dense IRC client (mainly because of the channel picker) if you want. And if you want to go really crazy use it in a browser and userscript customize it or use betterdiscord.
I think a lot of the features like embeds and emoji reactions add a lot of value compared to IRC (which I think is also why the IRC world is trying to add those features).
are those attributes now assets?
Pretty much. It's the survivability onion. You can't be destroyed if you can't be discovered.
Sort of, except if no one can ever discover a community it is always dying by default
Personally I'd love to find a decent online community these days, my social circle has shrunk considerably, but idk. It seems difficult to start fresh with new people nowadays
we were made to socialize in person. you can mimic it online and nourish existing connections over it but nothing helps build friendship more than being in the same place at the same time a few different times and talking to each other
Thats true but online content has always had its place. 25 years ago finding forums and irc was a god send, my lonely hobbies and interests became things i could regularly talk about. Its just modern social media abused the system, the algorithm, and us.
Which is all to say i agree about needing mostly irl, but there is also something of online community that irl could never replicate (for most people).
This shit will come to Discord too.
on the public servers yeah. but the ones im in with real people who know each other will be fine.
I think the problem is not keeping agents out of private real people spaces, but for people who dont have any pre-existing or 'real world' connections to these communities to find a way to prove they are a real person over the internet alone and get an invite
On a related note, I think this is going to be the biggest challenge to most folks when it comes in resisting using government ID online. it will be the apple offered for easy proof youre not a bot to normal circles.
It's already there.
I really don't understand the folks fleeing to Discord. A mailing list does 99% of the same thing for most of the communities.
Sure, if you want to chat while gaming, that's the whole point of Discord. Ganbatte.
But, for everything else, Discord is such a horrible misfit that I don't understand why it's the default.
> I don't understand why it's the default.
Because it equally well supports real-time communication.
And it looks shiny.
And some people use it to e.g. watch a video together, or other social purposes.
Discord is terrible. Full of bots, creeps and ai slopped to the gills.
Some communities are better than others but the sheer volume of stinky trash is immense despite discord and the poor volunteer moderators efforts to prevent it. Most mods are neutral on it too.
There are chat communities that are still somewhat safe with zero user verification. But I will not mention them.
There's this old meme where someone asks what will happen when AI bots posts helpful, curious and thoughtful messages!? That's mission accomplish :D They can't be better then the average human though because of training data, so I don't worry about AI comments getting up-voted by real humans, I am however worried about fake upvotes.
> They can't be better then the average human though because of training data
Is this based on the belief that an LLM can only represent an "average" human being?
It is not a meme, it's an xkcd: https://xkcd.com/810/
Doesn't help there is that feature that hides the user's posts and comments
> I do know for a fact that many "users" here are LLMs.
HN autokills comments it detects as LLM. I think maybe you're not giving HN enough credit. :)
HN front page is about 25% LLM written blog posts at any given time.
There’s no rule against submitting LLM-written—ahem, “cleaned up my notes”—articles, just comments.
>HN autokills comments it detects as LLM
No it doesn't. Unless you have proof.... ???
I have read enough “you are replying to an LLM” comments that I am pretty sure this is still a hit or miss process.
Why do you think those comments are accurate? Maybe those comments are by LLMs? If you believe crowd wisdom on its face, you will have big problems with LLMs.
It needs help. I often pipe my screed though an LLM and post it. I do request that it use a 10th grade reading level, and no emdashes.
For giggles, here's how it would look for this comment. Rather meta, but in this case it removed the "It needs hellp" so here we are.
I often run my screed through an LLM before posting. I ask it to keep the writing at about a 10th grade reading level and to avoid em dashes.
The question is how reliable that detection is.
Reddit sold it's data to AI companies for training[1]. They could have refused, but companies like OpenAI likely would have harvested that data anyways. As such, it should not be surprising that AI models are pretty good at generating reddit posts. They were specifically trained to do that.
This is sad, because Reddit remained one of the final bastions of human content on the internet. For several years, appending "site:reddit.com" to a google search was a valid way to get something usable out of a google search. Doing that is still an improvement over raw-dogging Google's ranking algorithms with an unfettered search, but AI slop increasingly is the result.
This is one of my great disappointments in the current rise of AI. LLM's can give good search results when dealing with a topic they've been specifically trained on by human experts, but they're not good at separating human-produced signal from AI slop noise. We've done nothing to prevent a sea of AI slop from being dumped on top all the human signal that's out there. When AI companies enter their enshittification phase and stop investing in expert human trainers, the search results LLM's produce are going to fall off a cliff. Search is a bigger problem than ever.
_____
[1]https://9to5mac.com/2024/02/19/reddit-user-content-being-sol...
> As I went through the posts it wrote I realized that as a reader I would have NO idea that these were just written by a computer.
I don't suppose you could show some examples? How convincing is the state of the art now?
There was a post today that Google introduced unbreakable capture that required unrooted phone to pass its QR code.
We may end up with things like that…
Communities in FB, WhatsApp, Telegram etc are actually flourishing. As it appears real time gated communities are doing fine.
It’s an unpopular opinion but I am looking forward to ID and age verified social media. If done right we can have real people around again.
BTW, ironically the harsher communities like 4Chan doesn’t seem to suffer from the dead internet. I guess it’s either because the advertising value is too low to justify AI use there or maybe AI API providers refuse to work with such a content this reducing opportunities to infest with bots.
> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs
People using LLMs without being fed their own post history are still pretty easy to detect. There's just something very recognizable about the cadence and tone of LLMs.
What really stuns me is that if you call someone out for it, 9/10 times you get absolutely buried in downvotes. Even here on HN. Its like people are angry that you're lifting the curtain on the slop, that the writing they enjoyed is fake.
Unless their account is <1 year I wouldn't assume they are a bot.
Reddit astroturfing firms and bot farms learned to buy/use “seasoned” accounts over a decade ago. I’d venture there have been countless bots just in a holding pattern harmlessly building up reputation and a human-like history of posts across different subs etc just to eventually be either activated or sold to someone else to “burn”
It used to be super common that when you spotted a bot post and clicked through to the user's history, you'd see very average, human-looking activity from years ago, followed by a long gap of inactivity, and then a flurry of obvious bot comments.
It's very obvious that these accounts were abandoned and then either bought from their original owners, or more likely bought from someone who compromised them, because of their history and karma.
And I would bet money that Reddit is well aware of this phenomenon, because not long after it became so common as to be impossible to ignore, they papered over it by allowing users to hide their history from public view. (AFAIK subreddit moderators can still see it, but typical users now have much less ability to see whether they're interacting with actual humans.)
That and locking down the API meant no more sites offering readily available visualizations of this type of thing
> allowing users to hide their history from public view
Yeah it's become my default assumption that any user who does this is either a bot or a bad-faith troll.
I recently spotted one unmistakable example of this[0]. It’s been a trick for many years now that duplicating a human post and its comments is a good way to appear human but this was quite the example.
0: https://wiki.roshangeorge.dev/w/Blog/2026-01-06/Is_The_Inter...
> duplicating a human post and its comments is a good way to appear human
Also just repeating something from the linked article, but often with different wording and in a tone that makes it seem like it was something that the article missed.
So what is the comment frequency of these bots? There must be some signal in the activity even if the comments themselves pass the turing test.
Even if there was, I doubt Reddit cares enough to go after them when it’s boosting their valuation
If you find one account you can find a few dozen spam accounts by building a graph of what posts they reply to
Most of them have private profiles these days
Does it matter? With enough you can just have them upvote each other.
So easy to purchase online accounts nowadays, neither karma nor age of the account matters anything anymore.
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
You can have both IRL and online-free-of-bots. I already wrote about it but one of the very best forum I'm a member of, where real people are posting, requires to be vetted in, web-of-trust (but IRL) style. It's a forum about cars from one fancy brand and you can only ever join the forum by having a member (I think it may be two, don't remember) who's already in confirm that he saw you driving a car of that brand. It's not 100% foolproof (someone could be renting the car for two hours and show up at a cars&coffee or take a friend's car etc.) but this place really feels like a forum of yore.
And people do eventually travel, so it's bound to happen that an owner shall go to another country, meet someone there, vet him in etc.
Now, sure, it may not be the "1 million users acquired in three days thanks to my vibe-coded app" scenario but that is the point.
You can imagine other domains where IRL communities have local groups, but where forums regroup different IRL communities all interested by the same hobby/topic/domain. And when people travel and meet, the vetted members do grow and connect.
Oh and on the forums a lot of the posts are pictures, where "Julian xxx" met "Black yyy Cyril" and you see both cars (and from more than two people): suddenly it becomes much harder to fake a persona... You now need to fake both Julian xxx and Black yyy Cyril and fake the pics. And explain why your car has never been posted by any carspotter on autogespot etc.
You can imagine the same for, say, model trains: "Met Jean at the zzz meetup, where he brought his wonderful 4-8-8-4 'big boy' locomotive, I confirm he's into the hobby, vet him in".
Naysayers and depressive people are going say it cannot work but I'm literally on one such forum and it just works.
P.S: if I'm not mistaken in the past in some nobility circles you had to be vetted by up to sixteen (!) other people from the nobility that'd confirm they knew you, your parents, etc. before you'd even meet the king/emperor/monarch to make sure that someone from far away couldn't come to, say, Versailles or Schonnbrun pretending to be a baroness or count or whatever. Quite the extensive check if you ask me.
Reddit was already on its way way before this LLM craze, hopefully the recent tech-related changes will only accelerate that process.
I find it amusing that this is the top comment. Reddit is so awful you finally wrote it off, but not before you used it to try to “karma farm and do some covert advertising”. It’s on-brand for HN hypocritical bullshit. But, since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard, have an upboat fellow traveler.
> since we are slamming on Reddit anyways without realizing how fucked HN is by the same petard
Same as it ever was.
> Online communities are definitely dying. I guess I hope that maybe IRL communities have a resurgence in this wake.
Good. Tech has been artificially propped up to generate liquidity in markets. Never had organic demand or traction. Most output was trite and meaningless; just capitalism.
Zero cares if all the HNers rich on tech stocks go broke.
Happy to project as little concern for their well being as this forum has projected at other's fields tech eliminated.
Some wild Dark Triad shit peddled here where there is no obligation to humanity's problems, just HNers own personal preferences.
Yeah ok that can be projected back, terminally online script kiddie; fuck your stock portfolio and I hope you end up in public housing assistance.
How do we know now that this comment wasn't written by LLM?
You don't and that's the problem :)
I feel you. Especially in the larger subreddita. i participate, and mod, a few small ones, and the community there is pretty strong and folks shut down ai slop pretty quickly.
I'm not saying being a mod means it's bullet proof, but i do notice smaller communities tend to self police better and know what's real.
That said, your experiment scares me as well.
I will say that I believe you probably have absolutely no idea because it's not "slop". It looks like every other reddit comment you see.
My experiment was focused on niche subreddits as well due to the nature of the product I was trying to market.
Reddit users are definitely smartening up on many subreddits to the same kinds of engagement happening.
This kind of thing made me imagine the creation of "digital towns" the other day.
Imagine an online community where you can only join on the recommendation of two other members, who you must have actually met in person, to participate. Meanwhile, you leave at least some of the activity publicly available to the general public so that interested parties can meet up IRL and join.
This could probably be implemented easily on top of existing online platforms like Discord, Reddit, etc. since it's really just a community building rule, not a community itself.
More of a philosophical question but if you have no idea whether it's a human or robot, does it really matter? Personally I dislike AI slop only when I can tell it is...
Yes, for a number of reasons:
- I am trying to learn about the topic at hand and trust a human's comment more than an LLM's guess - I am trying to connect with other humans to fulfill my social needs - I am maybe spending time to help another human out with a response because I want to help someone else - I am interested in the perspective of other humans
Those are just a few reasons. For each of those if it's actually an AI I feel I'm losing out on something.
It's easy to botspam Reddit because even the real users always acted like bots. The big subreddits were the worst, but contrary to how the users keep saying "it's good if you find the right subs," no it's not. Wrote that place off like 10 years ago.
> I do know for a fact that many "users" here are LLMs
What factual basis do you have for that?
It might come down to shareholder/IPO stuff but you can tell Reddit doesn't actually care to put the effort in to crack down on bots (however you'd do that) because they already don't give communities proper moderation tools/third party tools and the site does censor
Whatever allegiances (with people, or allegiances to ideas) Steve Huffman has, or people like him - it's not enough. It's a site seemingly killed by greed
(Yes, I know moderating this stuff at scale is hard)
- A human. Beep boop.
I wonder, how much of the discussions on the results of agentic coding is just LLM slop.
Do you have an example of comments people engaged with?
> where I had an agent karma
Was this a browser using agent? What did you use?
It used the browser agent to grab user cookies after signing in, then made API calls iirc.
Using just a browser is way too token intensive and slow. It would look for 401 errors then run the browser automation to login with the credentials and grab the token.
I'm surprised these platforms don't have advanced heuristics to detect API calls and inauthentic traffic.
Did you clone the Reddit API from browser traffic and then turn it into a 100% API driven thing?
I'd imagine they'd be sniffing browser agents, plugins, cookies, etc. to fingerprint. Using JavaScript scroll position, browsing rate and patterns, etc.
Maybe their protections just aren't that sophisticated.
Reddit is known to fingerprint TLS and quickly shadowban accounts that don't have the fingerprints of browsers.
TLS fingerprinting and Cloudflare are easy to bypass. There are lots of libraries that do so.
The application-layer stuff is harder. Each application can develop its own heuristics, and that's difficult to automate in a cross-cutting fashion.
Reddit doesn't do anything about that? That seems stupid.
> I am not quite there with Hacker News but I do know for a fact that many here are LLM's.
Please don’t do this here.
Don't do it anywhere. He's a jerk for doing it on reddit.
Reddit is the sewer of the internet. Good place for LLMs.
People live in and depend on that waterway. Just because it's beneath your standards doesn't mean it isn't vital.
You're giving "let them eat cake" energy.
I can assure you nobody in the world "lives in" nor "depends" on reddit to live.
Everybody lives in and depends on our shared social substrate.
Explain how that's different from Hacker News?
You shouldn't live in a sewer.
People are definitely trying to make HN bots because I have seen several get flagged. No idea to what end though.
People still see HN as a valuable place for promoting their businesses. Usually some poorly thought out SaaS.
If you look at what people outside HN talk about HN, it's not uncommon to see wannabe tech entrepreneurs talking about how to promote their products via Show HNs and how to stay HN front page. It's honestly a little sad considering that HN has a tendency to rip these projects apart.
the suits or suit minded people have realised that HN is good for advertising to the kind of demographic that'll give them free labour and is easily swayed by whatever the latest trend is
That's why we all love the MIT license.
Why would reddit bots exist? (In)organic advertising, same concept here.
The ones you see flagged are the very obvious bots. What about the more sophisticated ones? How do I know skupig isn't a bot?
Possibly to test reactions to a bot they plan to build a startup around.
I've seen some claim they do it to avoid stylometry or being fingerprinted, or because of social anxiety problems.
Some people just have a compulsive need to optimize everything, and HN's guidelines and tone policing are more easily followed by a bot than a human.
> HN's guidelines and tone policing are more easily followed by a bot than a human.
HN's guidelines aren't that strict and the mod hammer is a plushie. It's not difficult to get by here. It's also kind of useful for critical reflection/self-regulation to hear the occasional "you came in too hot" or "don't be boring" from a moderator.
Seems better to me to just try to be sort of reasonable and let the mods nudge you if they need to and let your comments be downvoted from time to time. What is the goal of these people, to never experience correction in their lives? To never write an unpopular comment?
> What is the goal of these people, to never experience correction in their lives?
Look at all the people who complain about cancel culture. There's a huge swath of people who don't ever want to hear "that was mean/bad/shitty".
>What is the goal of these people, to never experience correction in their lives? To never write an unpopular comment?
Yes?
That seems like a really extreme goal to me. I should hope there's a better way to address the anxiety or whatever it is that's motivating it.
I didn’t say that people weren’t doing it. I was asking this person not to do that here since it sort of sounds like they have plans to
He's stating a fact. Turn on showed in your options and scroll to the bottom of the comments on any popular story. There are so many agentic users here.
Having to turn on showdead (which I have turned on by default) demonstrates that’s it’s not much of a problem in practice.
I generally disagree, because the level of discourse here has always been very high, curious and intellectual.
It has, and the well prompted agents still give that. It's very weird.
I just don’t even understand the appeal of having a bot interact on forums for you unless you’re astroturfing for your company or personal brand or whatever
Maybe 1:100 comments match any one of those attributes.
Most comments are just grammatically "correct". Not a high bar.
The comments that aren't grammatically correct get pedantically corrected by those HNers too, so that's nice.
This site is CLEARLY astroturfed to hell and back and infested with bots. Any attempted discussion of this fact gets killed REALLY fast.
This part of the guidelines is a 15 year out-of-date bad joke:
> Please don't post insinuations about astroturfing, shilling, brigading, > foreign agents, and the like. It degrades discussion and is usually > mistaken. If you're worried about abuse, email hn@ycombinator.com and > we'll look at the data.
"We'll look at the data". Sure buddy. You'll do what you always do, which is apply to banhammer to anyone that's not following your talking points, and tone police the actual humans.
Enjoy "conversing curiously" with bots while the mods tone-police non-bots out of existence.
For what it's worth the admins here have let the tone of conversation slip a little when it comes to AI, as in there are many people who now openly mock (and worse) the AI zealots and there's no admin coming in and "saving" the metaphorical day anymore. In the not so distant past that kind of behaviour was almost instantly reprimanded, kindergarten-style.
So you ran an "experiment" where you deliberately made someone else's community worse to see what would happen? Cool project.
Unless you've discovered the secret sauce, LLM comments are very obvious. Even Altman revealed that they focused on coding at the expense of writing.
The obvious ones are the ones you notice
LLMs are not good at writing. If they were we would have entire libraries of new, amazing literature.
Exactly, they aren't good at creating new material. But many discussions in comment section are simply regurgitations of existing material, which they are good at rearranging. New novel discussions in places like this are actually a very rare thing, as many comment sections are simply people who already know informing those who don't. I'm doing that right now, funnily enough.
No, they aren't even good at rearranging existing material. They produce bad writing that only superficially looks good in a lowest-common-denominator sense, and falls apart under any close examination. Everything is wrong with it, from the sentence structure to the rhetorical forms to the substance. AI 'writing' is a loose collection of cheap tricks that score well on A/B.
Neither are most humans
Agreed, some humans are good writers, and no LLMs are good writers.
This is rather moving the goalposts from "plausibly human comment" to "meaningful literature", I think
No. I'm drawing it out to its logical conclusion.
It’s poor logic, a non sequitur. An absurd reduction. By your argument anyone who hasn’t written a great literary work is a poor writer, and would be bad at writing online comments.
LLMs aren’t lacking in the sort of writing skills that make for superficially good content. They know grammar, they know rhetoric, and they know their audience. You can’t tell them from a human on their writing skills. Where they tend to fall down is their logic and reasoning skills, and unfortunately it seems you can’t use that to distinguish them from the average online opinionator either.
No, that is a mischaracterization of what I wrote. They are great writers if you enjoy formulaic writing.
With the current batch of SOTA models, it is not hard to prompt a model to pass the sniff test on social media forums. If you don't believe me, try it.
All you really need to do is give it some guidelines of a style to follow and styles to avoid. There's also a bunch of skills people have already written to accomplish this.
I have worked with LLMs for a couple years at a very non-technical level and it was not that difficult to give it proper prompting and reference material.
If you are reading LLM content just about everywhere and have no idea. Obviously there are easy to spot things, but the stuff you don't spot is the stuff you don't spot
People that like to fancy themselves as good llm content detectors just end up accusing everything they don't like as llm content.
The only thing worst than a slop comment are the people that bitch about it incessantly. I'm convinced it's become a new expression of a mental illness.
The main thing I suspect of being LLM written is the sort of LinkedIn style: very short sentences, overly focused on sort of… making an impact on the user. But that’s also how a certain type of bad human writer writes. So in the end, I’m not sure I know if anything in particular was written by an LLM.
I guess… “that’s not just an AI red flag, it’s generally shit prose” would be how ChatGPT would describe most things nowadays.
It’s the distilled mediocrity of the statements. Never venturing beyond a 10% margin of what you would get if you sampled the opinions of 1,000 people who underwent jury selection by west coast liberals.
A mere opinion is not mental illness.
Was that written by an LLM? It isn't that it's a mere opinion, it's that when every word out there has to be scrutinized for the possibility that an AI output it instead of a human intelligence that it gets pathological. Am I an LLM with the right prompts set up to respond this way? I mean, I know I'm not, but everyone else out there is just going to have to trust me that I'm not.
I wasn't suggesting you have a mental illness for having an opinion.
More, commenting that just as bad as generated content if not worse is every thread where the top comment is an accusation and ensuing witch hunt.
So, no, having an opinion is not a mental illness. Feeling compelled to call it out and discuss it on everything one reads may just be.
The threads that have the top comment saying "this is AI slop" are nearly always about an article that is obvious AI slop.
Threads that aren't - like this one - don't.
If you need to tell yourself that in order to cope that's fine with me.
Which part do you disagree with?
I’m thinking that I may actually prefer undetectable AI slop to human comments like that. I do agree with your upthread comments.
On the other hand, I’ve been accused of being AI/bot and if I say things the mod doesn’t like and is not their favorite thing to hear I’m “flamebaiting” or engaging in personal attacks when pointing out specific things.
Frankly, online communities have been doing for many years now, when the censorship, anti-free-speech, tone policing mods and mobs started dominating online and America really did not have the self-respect or confidence anymore to enforce the Constitution online.
> America really did not have the self-respect or confidence anymore to enforce the Constitution online.
“Mods are Unconstitutional” lmao
> I do know for a fact that many "users" here are LLMs.
Name and shame.
If you look at the bottom of most threads here you’ll see a bunch of green username dead LLM comments. Those are just the obvious ones though.
I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political". This is somewhat related to the Overton window but really a bunch of (mostly conservative) ideas get normalized so aren't deemed "political".
I see the same thing with "AI Slop". Yes, there is AI Slop but (IME) it's pretty easy to spot. But what's more annoying is how often people are willing to throw that accusation whenever someone takes a position they don't like, much like the "political" label. It's lazy and honestly just as bad as the slop itself because it unintentionally launders the slop in a "boy who cried wolf" kind of way.
I also have a theory that some AI slop isn't inherently successful. It's just heavily botted by people who are interested in promoting certain positions. I bet you could make a pro-administration LLM bot and another one promoting a communist revolution and no amount of model tuning would make the second as popular as the first because the first would hit third-party botting as well as platform content biases (eg Twitter).
I've personally been accused of being a bot. This is particularly true in recent time as I've tried to share facts and fact-based analysis of, say, what's going on with crude oil markets, the military operation in the Gulf and the politics and economics around it. I even saw one hilarious comment saying (paraphrased) "the bots are getting clever and posting about unrelated topics". This was funny because it never occurred to this person that no, it was just a real person posting something you disagreed with.
> I've been on the Internet for decades at this point and one thing I've noticed is that communities that, for example, ban political topics actually mean "positions I don't like" as "political".
This happens on HN all the time. For a lot of downvoters and flaggers, there are two kinds of opinions: "Things I agree with" and "Too political for HN."
> I am not quite there with Hacker News but I do know for a fact that many "users" here are LLMs.
This just makes me wonder...so what?
Some of the oldest posters here with the most karma continue to post absolute garbage takes on topics ranging from US healthcare to history of USSR, that are trivially disproven by learning the very basics from a Wiki article (e.g. not a high bar).
To be fair, this opinion slop is also present for new users and LLM bots, but is one kind really worse than the other, if both of them contribute to killing the community?
We already know what kills communities. It's the eternal Septembers. Infighting within leadership also doesn't help, but time and time again it's the influx of too many new users that nosedive and drown out quality contributions.
Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”, and then actually spend time reading them?
No? I’m imagining not at least. Because there would be no point to it.
If you would enjoy it, then I’m surprised you’re here and not just simulating the experience with your LLM by yourself.
> Would you enjoy the experience of telling your LLM “make a HN-style comment thread on $subject with 200 comments, no trolls please”
The reason I'm not simulating the experience with an LLM is because:
1. It costs more time to do so, because I have to prompt it to create a single comment. Multiply that by the typical number of an HN thread.
2. I suppose in a way you need bad takes to form your own view of a topic or an issue. LLMs would also be unable to provide truly unique experiences, such as some of the veterans who sometimes post here who were part of the living computing history as we know it.
> I’m surprised you’re here and not just simulating the experience with your LLM by yourself.
That's something you imagined that I claimed I want. If you read my comment again, you'll see there was no such thing.
An irascible human being with "wrong" opinions is still better than a polite and factually correct bot because there's no fucking point in having a conversation with a bot. We're here to have conversations with people, not to prove fact beyond a reasonable doubt.
Do you really not care one way or the other? Would you really rather just be talking to LLMs here? Or would you just script yourself as well and call it a day? Then what?
> We're here to have conversations with people, not to prove fact beyond a reasonable doubt.
Maybe you are. I like getting to a reasonably correct model of a topic or issue. Bad human takes can still be useful here. I just get inevitably tired of the people crying about potential LLM comments all the time.
> Would you really rather just be talking to LLMs here?
Obviously we're not there yet, regardless of what I want. But there is a great number of HN threads posted here that touch on topics that have been discussed so many countless times, that an average LLM summary would do better than most comments.
Dead Internet theory ?
I run a niche creative community, and we outlawed AI-generated content in 2022 as it was easy to see how corrosive it would be to the community.
It hasn't been easy. We ban fake AI accounts daily and shrug off around 600 AI content creator accounts monthly.
It's a lot of work, extra work that wasn't needed before AI content came around, and of course, that is an extra cost.
I fear losing the battle.
What about charging $1 or $5 for an account? Seems like you could stem the tide pretty easily with something like that.
Or applying for an account could involve sending a handwritten letter by post.
We bringing back Something Awful, now?
It never left.
If you head to Twitter right now, the vast majority of bots are blue checks. It seems to actually encourage the opposite, where you trusting that someone paying $8 for an account makes you even more likely to fall for slop
I think twitter is an odd-one-out here, twitter as a whole has been heading down hill ever since the acquisition, and I wouldn't be surprised if many of those blue checks are officially sanctioned bots. Especially given the way so many of them push the same narratives that Musk does at the same time he does.
Unlike a lot of communities, yours at least started on the correct side. Better to ban outright, than to slowly realize that you should have banned it.
Indeed. Take a soft approach, or "wait and see", and you'll just allow your community to get infested with slop enthusiast crybullies that loudly protest any pushback against "genai content". The communities that draw a firm line and hold it will be the only ones that endure.
It was a surprise to us how vehemently some folk defended AI content and assumed it was their right to post it within our community.
We had no problems with people using it and posting elsewhere, it was the demands that we must allow it that were problematic and made us question whether we were doing the right thing.
No regrets now, though, as we see competitors being flooded with AI slop and they are too invested in it to change now.
Now I see it as the perfect tool for impostors.
>shrug off around 600 AI content creator accounts monthly. >I fear losing the battle.
I was in a small niche creative writing community for a while. Circa 2021\22. AI wasn't why I was there but I demo'd a few LLMs to a lot of the users in the Off Topic section because people were curious. Even with an explanation of how they operated, almost everyone was at least interested. One author told me how he operated similarly, rote learning how to write like his favorite authors by copying out their texts, hand written, word for word. Their concern was largely that they were too hard to use from a technical perspective.
These people knew I was there to learn, and that I was unlikely to ever try and publish LLM derived content. I said as much often.
Sometime in late 2022, a switch was flipped. And almost all of them started talking about how AI and those who used it were unambiguously evil. They didn't say my name, but they stopped engaging with me. Gradually, they started reposting twitter content from extremely anti AI people. Complained about AI submissions to various publications. Eventually, someone reposted a tweet calling for the death of anyone who used an LLM, with not even a single disagreement (and lots of encouragement)
I just bailed. I had only ever engaged positively, answered questions for the curious and tried to help people out. I posted one AI assisted story, and that was to demonstrate how my contributions were tracked vs AI contributions automatically in the editor to satisfy someones curiosity. Clearly highlighting the bits I had written. Just a technical demo. No one was asked to enjoy or positively engage with it as if it was human written.
A while later, most of their submission rules were updated with a new clause, if it was judged that AI written content was discovered, they would blacklist that person from all submissions across their entire community. Considering I had demo'd LLMs, and the uselessness of AI detectors, it was clear to me that these people would be able to justify blacklisting me if I poked my head up at all. I had been developing my own story for submission (myself, no LLM content), but I just dropped it. I just didn't feel like sticking my neck out for the witch hunt.
I also used to be quite engaged with blockchain. And it went through a similar process, most people ignored it until that paper about the power usage (Claiming it would spike to some level it never reached) and then suddenly being associated with it was an outrageous moral crime. But after a while, when it turned out that the power use claims were largely a nothing burger, people gave up on the hate parade.
I don't think you will "Lose the battle" (at least in terms of keeping AI users out). And its always ok for small communities to be selective about their membership. I just don't think its possible to maintain such artificial rage for more than a few years. The AI Datacenter water/power claims are a clear London Horse Manure problem that looks set to resolve itself, and the copyright issues will get sorted to some degree. Eventually I think you just wont care enough to ban anyone except low effort spammers (of which there are a huge amount, granted).
YMMV
You're a victim of the uni-cause
I kind feel this might be good. Bot writen comments and AI media that can no longer be distinguish from real, will make us human leave the social networks, which helped to separate Us humans. Going back to the real world were you can trully believe on what you see, and enjoy the tone, look and scent of of our fellows humans beings.
This seems naive. As long as people are "enjoying" the AI-infested social networks, or at least not annoyed enough to leave, they will stay on them, and become further disconnected from reality. We have half of EU teenagers talking to chatbots regularly. Alienated people flock to them.
Social media that caters to what the user wants/interacts with can become infinitely more so. This is already applied to entertainment content across tv and the internet.
At some point an instagram/tiktok/etc user could see nothing by real people and not even know what is promoted vs ad vs post.
For one point, I was a daily active user of Reddit for 10 years, I deleted my account and left the platform in January over LLM content.
Actively seeking out a chatbot is different than wanting to talk to humans.
A few tech companies managed to get massive numbers of people addicted to toxic social media content that was terrible for mental health but made a small group very wealthy. I don't think those same businesses and execs are just going to pack up and go home with an even more powerful content tool available now. LLMs are going to be used to create skinner boxes that make Facebook and Twitter seem like wholesome communities.
The problem is that many of us have niche interests and no one local to discuss things with or get made fun of for being a nerd.
I loved maps and geography as a child and still do. I've never met anyone in real life that likes it as much as me. But on the internet there are places were I can discuss it and other people share fascinating articles, pictures, etc.
But I have a lot of friends online, both ones I made online and ones that have moved away from me and vice versa.
I don't want to be limited to only the friends I can make who live near me
If it can no longer be distinguished from real, why would it make people leave?
Problem at scale. Doesn't matter if someone is consciously able to identify individual bot accounts or comments. There can still be a strong general feeling that something is very wrong. Leading to more and more frustration and unhappiness.
"Popular" reddit posts and subreddits are a good example of this.
It might be hard to recognize an individual user or post as AI, but it's not hard to recognize the negative effect in aggregate
One of the paradoxical things that makes me hopeful is that there's going to be such an incredible amount of low effort AI slop content that it's going to drown out the low effort human-made content and generate a large amount of distaste for it. So much will be so bad that good taste and high quality will be rewarded more status as the people who will say and believe anything will be led astray and left behind.
Maybe it's hard getting across what I mean so a more concrete example is there will be SO MUCH clickbait out there that serious outfits instead of being forced to do it will be able to successfully differentiate themselves by NOT doing it. (and many similar things in different arenas)
I'm trying to say that LLMs raising the noise floor will drown out a lot of the toxic noise that's been plaguing us.
I can hope.
> So much will be so bad that good taste and high quality will be rewarded more status as the people who will say and believe anything will be led astray and left behind.
I really want to believe this will be true. However, I also suspect there's some external driving force, that I cannot readily name, which is making people incapable of consuming anything except this low-effort content. I mean, obviously it's working to some extent. Perhaps AI will be the thing that accelerates its death, but part of me thinks something else needs to happen beyond just an increase in useless content.
In my opinion there isn't an external _nefarious_ force causing all of this. Certainly those forces exist but without them much the same would be happening.
It's the economy of everything being free but supported with advertising. That mechanic is what leads to the race to the bottom lowest common denominator human motivation hacking attention toxicity. (yes that's a bit of a ramble).
If people weren't getting paid for the smallest increment of attention they could grab, it wouldn't be promoted the way it is. I don't have a high opinion of the things which grab my attention, but they still manage to do it sometimes. I think many people are in that boat. If there were other mechanisms with which we rewarded people for doing things, something different would be optimized.
And people just wouldn't reward the 10-second-gratification in anywhere near the same way if it weren't for the advertising.
Have you considered that (further) lowering the signal-to-noise ratio will make it much more difficult to find and distinguish a signal?
Yes, but I'm hopeful for a survival of the fittest instead of an extinction.
Now there's more pressure to have a stronger signal and hopefully rewards to match.
What do you think happens to the least prolific organisms that lose the survival of the fittest?
To be clear I'm talking metaphorical survival of the fittest that takes the form of prestige/popularity/status/etc. of people and organizations.
The important thing to recognize is that quality of content has never been the driver of online communities. As long as they provide an engaging break from real life, they will exist and thrive. I think the negative association with LLMs is a phenomenon that will die out in the 20s. Our understanding of authenticity will evolve and so will the tools and platforms. The internet has always been extremely artificial, that won't change very much.
The balance is so far out of whack with LLM's now in online communities. People crave human interaction with like minded individuals, and whoever figures out how to give authentic online experiences is going to be successful. Maybe small communities need to come back, where you build credibility slowly. Why does every site have to be a monstrosity that wants to build a hundred million users to IPO. It just attracts the worst. I was active on Reddit for years under the same username I have here. I have pretty much abandoned it.
> People crave human interaction with like minded individuals
I don’t think they crave it enough to make a difference. Even before AI slop, Reddit had made successive changes that led to much less of a feeling of interaction with real, authentic humans who could become your buddies. The UI de-emphasized usernames and hid the sidebars where subreddits could have their own distinct community atmosphere. I hear that now on comment threads, Reddit will even hide a decent number of posts from other users, so that a poster may well be talking into the void.
It is on old-school fora that one can get a sense of actual interaction: with avatars and other personalized touches it’s easy to gradually learn who is who, and there is a culture of longform text where you can actually get a sense of other people’s personalities. But how many people under the age of 35 or 40 are joining those fora that survive? Give people a choice, and it turns out they prefer the dopamine hits of engagement-maximizing commercial platforms, and the smartphone as the default (or sole) interface to the internet with all the death of nuance that spells.
Some definitely enjoy the dopamine hits and get addicted to the doom scrolling. Maybe I am just too old to understand it and the internet is passing me by. Some of us still like conversations like this. Real conversation in a respectful manner even when we question each others viewpoints. The old internet is still there in some places and I'll continue hanging out there as long as it does. While I have great friends in real life, not that many of them are old tech nerds, so the internet is really the only place to talk to like minded people.
The company I work for has a deep rooted community side and despite what big techs do, I am 100% confident the only aspects we have in community features are for the user benefit. No gray area. Just that.
Since the AI sloppification we lost considerable amount of traffic to bots. But worse than that, we lost users who tended to contribute back with others.
We can leverage multiple ways of exposing community data to members, so it is not that we are loss because of that, but more in the fact that we have 30y or so of good feedback on how the community around the platform was good for people and now everything is at risk...
Don't get me wrong, my work is work... There are premium features and else, but the amount of value one can get for free is what the platform is known for. And we know many people use it for free for years and when they need or can they subscribe and mostly stay for years and years.
The fact people are losing those connections is depressing to me
I left multiple online communities because the slop and the slop users were unbearable.
I use ai okay. I think it's useful. But people who dove hard into this stuff treat all text on their screen like it's a chat bot and not a person.
"Rewrite this code using the new API" "excuse me?" "Can you do it I need it right now chatgpt won't compile!" "Show me your code please" provides the biggest pile of dookie ever "hey can I ask how you came to decide on any of this? Maybe we should rewrite what you have here because x y z is concerning" "the ai did it I am learning. There is no need to rewrite anything just write this section for me" " no thanks" someone else does . user leaves
I've seen people like this 15+ years ago on #learnprogramming on Freenode, I'm guessing LLMs just tend to validate that behavior instead.
It adds a depth of nuisance that's for sure. I've seen users talk about how they can't wait until they don't need to ask for help anymore and can just use LLMs. Meanwhile I'm directly messaging with the person who made a package and asking why they designed it the way they did beyond greatful to learn 8 new things in 6 sentences.
Mostly on IRC people either learned or got told to leave. There was significant pushback on people just looking for someone else to do their homework.
AIs have changed the feedback loop here such that these approaches are rewarded and even lauded.
Sadly the imperative is, as often, a call to everyone to be good guy and make less noise. Unfortunately, it doesn't work, neither at personal level, nor at global.
One may be quiet, but what if your friend/acquaintance/fellow got possessed by some AI slot machine, and is sharing his "products" enthusiastically? I had such case, and right from the very beginning was dismissive and rude, and it doesn't work -- he keeps sharing various artifacts.
On a global level, yes communities die out. I think, global communication has reached the point when it's more a liability than a benefit. In late '90s and early '00s, maybe until early '10s, getting more connected could lead you to nice clients, getting hired etc. Nowadays, even before ChatGPT 3 in '22, every such area became overcrowded, underbidded, etc, and LLMs, surprisingly, added not much new -- just augmented this trend.
Giant online communities, yes. Small ones seem totally unaffected afaict - some harder to spot scam/spam accounts, but they're outed as soon as they act. And any invitation-based thing should almost perfectly block those.
Smaller communities are generally a lot healthier anyway, so tbh I don't think this is all that bad of a thing. I don't think it's possible to be open to millions and also be healthy, unless you spend a lot of money paying moderators (and regularly rotating them, to prevent burn-out or mental harm from too much exposure, which ~0 do in an even slightly ethical way).
A few things. A web of trust of some kind like vouching may come back, and general algorithmic silencing of low quality members. Also most governments are going towards the South Korean model of government-verified ID to post online to keep teenagers off social media. The same tool can be used to greatly reduce spam and slop, if that's what platforms want.
Also people will get used to AI in online spaces as AI quality improves. If I'm online trying to get help for some task, I personally don't care who wrote what if it is correct; it's not like humans have great track records of accuracy or substantial contributions either on average. Correctness is expensive in general.
If I'm online trying to relate to other humans emotionally, well I get what I'm paying for. It's been true forever that the better the gate, the better the community. I've tried to push the boundaries of openness, but as I've written extensively on MeatballWiki, soft security depends on there being more good than bad apples in a community. With machine intelligence, the economics of that are silly.
Regardless, people love people, so we'll figure it out. I'm optimistic we can rise to this challenge.
> I built a homepage on Geocities, complete with...a web counter
Yes, but how many decimal places did you optimistically give it, only to never use more than the "10s" place?
Entering the AI era, it's hard to tell the authenticity of things on the Internet. But sometimes, having a conversation with AI is not a big deal as long as we can gain something from it
1. human verification for auth.
2. only human generated input composer, no copy/paste, no file uploads ect. control the composer. control the camera sessions for photos videos.
3. no algorithmic feed that is designed for ad-spend and eyeballs.
4. moderate
You're absolutely right!
I've found the smoking gun ⸻ it's not your work, it's your prompt.
I've seen en dashes. I've seen em dashes. What kind of dash is that?!
It's been a personal favourite of mine to sprinkle into replies to clearly LLM generated textual diarrhea, it scores a laugh like, 1/10 times haha.
A three-em dash. TIL.
Mmm dash.
I believe it’s called the chungus.
There has to be room for an AI-driven project that expresses a unique idea, even if there's no community around it yet. Someone has to express it, and from now on that idea will largely be implemented with AI.
> A good use of AI is when it enables people to do something they couldn’t do before, to contribute to a community when they couldn’t before.
I agree 100% with the novel contribution aspect. But there's some nuance there.
For example a project might have no active contributors. It might not be something you can drop directly into your codebase. Neither of those is inherently bad.
As AI becomes more responsible for higher-level planning decisions, the value of an OSS project becomes less tied to visible community activity like PRs and issues.
I notice this in my own work a lot. I might not use that project's code directly. But I think about a problem differently as a result. I often point my agent to existing OSS projects as inspiration on how to solve a problem. The project provides indirect value by supporting architectural decisions, deployment approaches etc. Unfortunately OSS activity doesn't capture this.
> A good use of AI is when it enables people to do something they couldn’t do before, to contribute to a community when they couldn’t before.
There are two separate things here that are getting silently conflated.
> A good use of AI is when it enables people to do something they couldn’t do before
This could be good on an individual level, if say, a doctor wants to vibe code an app of some sort for his individual practice.
>to contribute to a community when they couldn’t before.
This is where it goes off the rails. If they couldn't meaningfully contribute before, they aren't going to suddenly be able to discern that whatever slop they want to contribute is of value to the community. That's just another way of saying, if I wanted an AI opinion on something, why wouldn't I get it directly from the source, and write the prompt myself, instead of have some intermediate human prompt the AI for me?
The human has unique context. They may work in a niche domain or they talked to people and observed an unsolved problem. Then they express a potential solution via OSS. It's like product sense. Then they share that with others who find it interesting. The code is a great way to encapsulate the idea. It is usually the result of research and back and forth not a single prompt. It would be way harder to think through or build a solution without AI even if they had context.
Because of the convenience. Why should I have to go and spend my time prompting an AI, if someone else has already done that for me. Same thing with food. I know how to cook a chicken risotto, but sometimes I like having someone else do it for me.
Who is going to verify that an AI-driven project is a unique idea? How do you distinguish between a genuinely unique project, a grifter who is shilling their "unique" project, and a new enthusiast who is convinced their project is unique, but is not? This is an impossible moderation task. The only options I see for a community are to either totally ban AI-generated content, or be totally consumed by it.
I don't really know. Certainly we need a higher bar. The Kafka example in the post may be hyperbolic but I agree it pollutes the space. But we also can't swing the other way and rely completely on out of date proxies. If you ban AI code there will be very little code to see in a year. It'll take time but we'll arrive at new norms. We built semi successful ways to filter content farms in the earlier internet days. The signal has to shift to "did they think hard about this problem" which has some observable properties. Like how they articulate the problem, or why it became important to them.
I have pondered the sensibility of using AI to support the initial birth of new communities. Given the needed social validation of seeing both 1. A populated community and 2. The tone of the community being grounded, non-toxic and useful.
The alternative is having a community born that will be small, have early adopters who can be overly passionate or critical and gatekeep folks from discussion. That means high effort to curate initially.
The “slopification” of the internet has been happening for years now but I honestly don’t know what a real solution would look like.
Most people aren’t willing to go through a identity verification process, or pay to join a community, and invitation-only spaces would probably lose diversity of thought pretty quickly.
Even still, I guess one of the above is a lesser evil because the bot problem is only going to become more unbearable.
P.S. Props to the author. I really liked this writing style.
"Build with AI."
No, I don't think I will.
I think one of the big differentiators of slop is critically reviewing the work as a human and continuing to refine before letting it see the electrons of public places. I think that the true benefit is harnessing AI to augment ourselves as an extension of us, rather than a replacement. I'd be curious if there is a way to effectively prove something is human first rather than AI first on the internet. I haven't figured out any particular way yet as even using AI to detect AI would require a sufficiently large sample to determine. Something that keeps me awake at night.
Online communities died when they were monetized and open
When LLMs were new on the scene, I thought trust would fade in the written(text) medium. I saw it happening on Substack, Medium, and Reddit. But then VCs pumped so much money and AI has gotten into every other modality (audio, video). The only thing I really interact these days are the human beings sitting in front me, phone calls with people I know and hackernews. Life seems sorted but something feels missing as well.
Edit - I am not anti AI but it is slowly killing the digital human interaction.
At this point, I see no identity verification or proof of some kind of humanity working.
I think what we need is the equivalent of what was done for CORS: client/server cooperation.
That is, APIs should mark that they are human only, and harnesses should cooperate with such flags and prevent calling said APIs.
It's not perfect, as it's client side enforcement, and one could still theorically build their own harnesses without, but that's the only way forward.
The sad part is that the cost gets pushed onto the good participants. Once enough replies feel synthetic, real people spend more energy deciding whether the conversation is worth joining.
Question for web devs - are captchas effective any more? If Reddit required a captcha on every comment, would it actually decrease bot comments?
There's a reason Google is switching to "scan this QR code on your phone with a Google-authorized TPM" kind of CAPTCHAs
I've never seen a CAPTCHA like that. What's it used for? Google Cloud services?
https://news.ycombinator.com/item?id=48039362
https://cloud.google.com/blog/products/identity-security/int...
It's a reference to this I think
https://www.androidauthority.com/google-recaptcha-play-servi...
Wow, that's bad. Looks like the warnings about TPM and remote attestation being a backdoor to total digital lockdown from the Stallman contingent were right.
The tin foil hatters are always told they're making slippery slope fallacies until they're proven right a few years later. Over and over again.
Sometimes. Other times they're just wrong.
It’s only just been announced: https://news.ycombinator.com/item?id=48039362
> Question for web devs - are captchas effective any more?
They’re effective at annoying humans. Driving traffic away from your site. Reducing conversation rates.
And training LLMs.
I was on Usenet starting in 1991. Once the Internet got popular with the general public around 1995 things started going downhill. Spam overwhelmed Usenet in the late 1990's and made it almost unusable for general discussion.
Stuff started moving to web site forums which I still don't think are as good as a Usenet newsreader. slrn was my favorite.
Then reddit came along and a lot of online forums started dying as people moved to reddit.
Just this morning on reddit I reported 4 separate posts as AI slop to the moderators. They need to change the categories as I flag it as "disruptive use of bots"
For 2 of the posts the moderators agreed with me and about 5 hours later the posts were removed. For the other 2 the moderators haven't done anything.
It's a losing battle.
Some of the posts start by asking questions like "I was thinking about this and... [long rambling paragraphs] Your thoughts on this?"
I waste a minute reading then another minute skimming the rest of it and then realize I wasted 2 minutes of my life. Then another 30 seconds reporting it to the mods.
This has exploded in the last 6 months.
Then there are all the repost bots farming for karma. Some subs have a rule that you can't repost something in the last 30 days or 6 months. But it is really ridiculous when something get 500 upvotes and then literally the next day a bot reposts the same thing and it still gets 300 upvotes. I think it is just a bot farm upvoting stuff.
Like many modern woes, it’s a problem of trust.
The baseline level of trust in an online interaction has been eroded significantly by LLMs.
The question is, how can we reverse this trend and increase trust?
I have a sneaking suspicion that it would help enormously if the stock prices of the largest companies in the world were not tied to how effective they are at hijacking as much of humanity’s time and attention as possible.
Maybe the fediverse can (eventually) help? It’s been a while since I looked at it.
Let’s empower people to effectively have more control over the content they interact with.
Social dynamics can make this difficult. We all want to be in the loop. The recent striking successes of the movement to ban phones in schools gives me hope.
> Maybe the fediverse can (eventually) help? It’s been a while since I looked at it.
The fediverse has been around for well over a decade in some form or another. It never caught on with society enough to make a difference. And unfortunately, the fediverse has now developed such a distinct culture of its own, Highly Online people with distinctive political and social shibboleths, that it even alienates many tech idealists around the world, let alone the general public.
The general public isn't alienated from the fediverse because of its distinctive political and social shibboleths, the general public simply doesn't know that it exists.
As far as the "tech idealists," a lot of them seem to want every space to be 4chan where they can be racist trolling assholes without consequence. And those folks have Nostr.
I think what could work is requiring users to prove their authenticity and uniqueness using a national ID of some sort. It would be bad for privacy, no doubt, but it surely would work. But the users' actual names should not be displayed.
I was thinking about that. It should be possible to do this in a way that mostly preserves privacy.
Sites and apps don’t need your actual national ID, just to know that you have one. I think it could be possible to have 3rd party verification services that don’t know where the verification request is coming from, thus preserving privacy on both sides.
I feel the root of the problem is that Google and major platforms defined "correctness" as "high impressions" and "high engagement." This created a game where AI-generated "slop" becomes the ultimate winner. For those of us trying to create or find constructive, deeply-thought-out content, the situation is becoming increasingly dire.
It is exhausting to see a single, sincere sentence based on genuine human experience buried under 1,000 pages of SEO-optimized, AI-generated "void" that Google deems "correct." Despite this, I will keep working on filtering through the noise today.
I feel that a lot in my side projects: maybe one should keep the half-baked AI repo for oneself and rather share what the experiment, the thesis, and the learning from the building are. No one cares much about the (un)finished product, as it can be replicated better in most cases with a couple hours' work of claude coding.
For instance, I really liked how Karpathy shared a high-level idea on the LLM-based wiki. It was sadly followed by a long tail of no-one-cares-about "Here is my LLM wiki product" posts pointing to the generic LLM-generated landing page.
I want my future community apps and sites to build in bot a flagger. I don't care how hard it is, the community that gets this right is the one I'll jump ship to.
Unfortunately I think reddit themselves encourage and want the bots because it leads to more traffic and ad displays so they make more money.
I've been messing around with a decentralized social network where you only see who you choose to follow.
It's implemented for plan9, but clients could be made for any OS:
https://youtube.com/watch?v=q6qVnlCjcAI
It sucks that the narrative framing device of 'human slop' has vanished in the last year. Some subreddits, like all location subreddits, lifestyle subreddits like malefashionadvice and redscarepod and entry-level academic subreddits like math and criticaltheory were already just hives of human slop before AI came around because of a structural design to the site that had the side effect of normalising a total absence of quality control.
Upvotes are not a good mechanism for quality control in any way because they force good content to have the same metadata as the content that is technically well-constructed but is irrelevant, meaningless, just a platitude, too obvious to be obvious or pablum. Upvotes turn everything into a shock-value dominated 101 space.
Online communities that allow upvoting / downvoting have been effectively dead for a long time because it's easy to manipulate conversations by elevating and punishing comments to fit a narrative. This is especially true on HN.
Ironically, aggregator communities like reddit came about because forum communities were dying off. Memeing about the latest news injected "life" (if you want to call it that) into the internet. AI is just taking out the trash, in my eyes.
On the other hand I think you need reputation mechanism. The biggest problem of online communities is that every moron (or a bot) has equal voice. Clearly democratic upvotes/downvotes don't work very well though. Someone who solves it is going to be the next billionaire.
I wonder when things get so bad that we end up filtering content made by accounts created before the release of ChatGPT.
There are already some sites where you're much better off if you have an old account. Like I have a super old Twitter sitting there for random stuff that requires it. I tried making a new one a few years ago, didn't post anything, and it got banned within 2 days for "bot activity." The old one has never been banned.
It was also so much easier to make a dating app profile back when I was single, like one click. Recently was watching a friend set one up, and now they not only want like 3FA but also proof that you're a human. Assuming the old accounts are grandfathered in.
Won't help much, botters can just buy old accounts.
Gets harder with all the 2FA stuff involved
Infinite new accounts vs finite old accounts for sale. Low effort vs high effort. It would help.
AI is lifting the voices of the lazys and below average to average people. for those who would never have progressed it might seem like a god given gift. for the ones with the desire to grow and learn and go beyond average... this is a curse.
Took the words right out of my mouth.
I'll remove the particulars to avoid anything partisan, but:
I failed to truly appreciate how cooked reddit was with bots until I accidentally clicked Popular and stumbled upon a national subreddit post with a 'chad meme', starring a particular political leader, whose unpopularity is hard to adequately convey to foreigners.
It was not just that this post had been so severely upvoted, but the comment section itself had a mantra more or less, with very little actual conversation, just echoing the same sentiment; and all those comments in turn upvoted to the point of drowning out the lone comments at the bottom (not downvoted, just not upvoted) expressing "???". I don't know if I'd ever even written the word 'astroturfing' before expressing my bafflement at a friend, so I don't think I'm very tinfoil hat about these things.
It was just utterly bizarre to see someone who can barely get a single win in public discourse being heralded -- monotonously -- like he was the second coming.
>I failed to truly appreciate how cooked reddit was with bots until I accidentally clicked
For me it was a wholesome response. It seemed genuinely kind/human.
Click on user profile...it's a bot just pumping out posts like that. Looked organic when seen in isolation, but when you see a wall of them you see that it's got to be an LLM (with a good prompt).
That was disheartening...I had kinda accepted that the sht-stirring rage posts might be bots but the kind comments too? Ouch
This is a good thing. social media was already slop before AI. If this gets more intellectuals off these same websites daily and instead spend their time to better things, then I love AI slop’s purpose. There’s more to the internet than Reddit, TikTok, and youtube. Really there is, if your circle of friends is small or non existent without going to the same dotcoms, you have an issue that is worse than any AI slop tbh
getting people off the internet is antithetical to the business goals of the AI companies. they won't let that happen without a fight
They're making it happen whether they like it or not.
Invert the economics. Right now there is value in posting LLM generated content that is more than the cost of using the model.
If platforms had a subscription model that you had to pay for in order to do more than just read comments, there’d be a lot less LLM content. There would also be a lot less of all content. But maybe that’s the price you pay (literally) to get rid of AI slop.
Re: "The Asymmetry of Bullshit"
I'm gonna speak on behalf of language models' capability of making online communities better. In recent times, the frustrating forum phenomenon of "learned helplessness" is making me too annoyed to participate. Even in a fantastic subreddit as /r/LocalLLaMA, there are people posting replies in the vein of
> user1: please help me understand this acronym the post title speaks of > user2: (explains in detail what it means)
In the "good old days", a low effort, surface level question would result in someone either muting or banning the person to keep the discussion high quality.
There I am, browsing a forum dedicated to LLM enthusiasts, and an unbeliavable number of people are asking LMGTFY/RTFM-level questions they could even find an answer to from a free Google Search AI summary, and people are rewarding them by actually responding to them with effort.
Thanks to models being quite intelligent at answering basics, the ban-hammer should be used more swiftly if people keep polluting forums with low-quality posts. There's no need to feel bad for them not having the time or capabilities to read through years of forum posts to feel qualified to answer.
Maybe even these sloppy posts authors can be outright muted or banned with a heavier hand for the sake of quality.
This kinda thing makes me sad that keybase sold out to zoom and wonder if it can be resurrected. It was such a simple web of trust that went viral enough that I still occasionally see it on HN or Twitter profiles even though it's been long dead.
There are maybe 20 or so online handles I know, some of whom I've met in person, who I deeply trust. To the extent that I fully trust anyone they vouch for too.
Even with just one degree, that's a large enough international semi anonymous online community that can provide value to each other through online text based communication. Doesn't need iris scans or credit card checks, just "patio11 on hn Twitter and whatever his domain is is one of the good uns" and a network effect from there.
Already seeing some form of this reputation staking in eg Pi PRs, everyone is treated as clanker slop by default but the entry bar remains quite low to prove and build reputation.
I don't think online communities will stay the same in the face of AI but I do think whatever comes next will strongly rhyme
I made this point elsewhere, but people are learning a lot of what us had to learn the old way which is no one cares about your stuff for the most part and now the value provided has to go way up to get people to care. That is, as the author says, the novelty has worn off and since we know it's AI the perceived value is also way down.
We're all recalibrating.
I do really think this is just a quick period in time before most people realize that the slop posting doesn't help them personally get anything and most give up and we go back to roughly the ratio of cool things with real value to see but like on a bigger scale because AI helps you do more as one person.
I don't know... I might have said the same thing about email/text/phone spam but it has only proliferated to the point where it's a constant stream of garbage. Email, text, and phone calls are almost completely useless at this point. Sifting the signal from the noise is a non-stop effort.
I think people who want to push a certain narrative might just set up a quick bot and tell that bot to start posting on Reddit or whatever and just let it run. Why not? Little effort on their part and they might actually have influence. The same reason why spammers apparently think sending me 10 text messages per day about a loan I've been approved for. It probably does work 0.0001% of the time, but that's okay if it's all automated.
I mean I think the dynamics are a bit different in online communities at least for actual communities and not drive by subs like r/technology or whatever.
Especially say here on HN with Show HN and such the forcing factors are "i get no votes or community recognition"
But I don't entirely disagree with you I think things won't totally go back I think it will settle way more than now though especially where things are a little more niche.
> I am not an AI-hater. In fact, I think AI-haters are on the wrong side of history. Incorrect.
What online communities? Ever since Reddit went all-in on censorship, actual conversation moved to the deep web, mainly on Discord and other places invisible to search engines.
There's a lot of focus on tech projects here, but it's not just vibe written projects that are ruining communities now.
No, it's a problem with art, text and videos too. Reddit was already becoming a creative writing exercise in many ways, with infamous subs like 'Am I the Asshole?' seemingly being about 80% fiction labelled as fact. But now you don't even need to know how to write to flood the site with useless 'content'.
YouTube is arguably even worse, since AI led content farms are not just spamming the hell out of every topic under the sun, but giving outright dangerous advice and misinformation on top of that. I saw this video about medical misinformation by these 'creators' earlier, and it genuinely made me want to see them crack down on this junk:
https://www.youtube.com/watch?v=UEfCTCBDKIU
And there's just this feeling of distrust everywhere too. Is anyone on Hacker News human anymore? Is that Reddit poster I'm responding to human? Are the folks on Twitter, Threads or Bluesky human?
The scary part is that you basically can't tell anymore. Any project you find could be AI generated slop, any account could be a bot using stolen images or deepfakes, any article or video could be blatant misinformation put together as a cash grab...
If something doesn't improve, pretty much every platform under the sun is going to be completely useless, as is a lot of the internet as a whole.
I think people like the blog author need to realize that this problem can't be dealt with content moderation or users trying their best to be honest. You just get a firehose with an on/off switch, you don't get free filtering or moderation with it.
For every argument against AI slop, you will get a variation of it's the future, or I'm 10x more productive now, I've shipped 3 applications in 2 days, etc.
They won't stop talking about it and defending it. But I can't get anyone to share their amazing work with me.
There is a reason the Show HN projects that are mostly vibecoded don't get much response. It's because they aren't any good. Comments that are AI generated are hollow. Videos that AI generated a shell of their sources.
Obvious slop still makes it to the front page of HN, and sometimes farms GitHub stars.
These posts also usually get all these glowing comments from users who clearly haven't checked the code. It's even worse when authors get busted and claim "Okay, Claude wrote it, but the design is mine" despite clearly not understanding the output themselves.
Unfortunately, that makes high-effort projects less visible. The SNR will probably keep getting worse until slop can be flagged on HN.
The importance of good search engines and good discovery engines will grow even more.
Search engines got SEO'd to death decades ago
What good is SEO if people just read the Gemini summary at the top of Google and don't click the links? We have a chance at a real search engine again, now that there's no money in it.
Can such a thing even exist now? Any search engine algorithm can be gamed by AI.
I actually think the good old Page rank[0] is crucial because if the authoritative sources link to some website, webpage or content it means that particular item provides some kind of value to the entity that linked it. I'm also a big fan of metadata which can be used to describe web content and make the content more usable to the search engines and the Web users.
[0]https://en.wikipedia.org/wiki/PageRank
AI will be a forcing function that pushes people to go meet in real life to do stuff together.
Even if everything online is fake, events are not. So if people say they’re going to show up somewhere, there must eventually be a moment of truth. And then you can form high trust private group chats to keep talking together.
It may be hard for the current generation of chronically online people to adjust to that new reality, but the next generation of kids growing up can get used to this now, and eventually socializing in person will be natural again and the internet is for bots and weirdos LARPing as something they’re not.
Maybe, but the small groups that form out there in the real world will each be much smaller than the large group that stays and gets jerked around by the bots.
The large group will have to endure the manipulations that we've come to know and hate from the internet, but they'll also be better coordinated than the small ones. They'll vote together, buy the same sorts of things, have an outiszed influence on the global conversation... They'll define the de facto majority opinion whether or not they actually are a majority and whether or not it's authentically their opinion.
I don't think that's a good outcome. We need ways to get on the same page en-masse, if only to counteract the harms caused by whichever highest-bidder is currently using an AI horde to control the other group. Besides, we should save them from this abuse for their sake, if not for ours.
The internet is worth fighting for, if we abandon it entirely we'll be forever at a disadvantage against those who would use it to manipulate.
I agree with the second part.
Do we need a new carbon-credit style market for companies that want to continue putting out such slop and paying for moderators to remove the waste after it's been made?
AI Slop is killing the mainstream communities and the alternative communities are filled to the brim with tankies/nazi's (unironically).
Original online communities (forums/chat rooms) got largely killed by social media, now social media is getting killed by AI slop.
HN is in peril and I don’t think it is a bad thing. Or rather, I’d like to bring back the old chestnut: it’s a good thing.
While the site has moved to using /showlim, the AI garbage just bypasses that and goes straight to the home page. Almost every project that’s being shown is vibe coded and looks exactly the same - generated by Claude or the like. This is an excellent test for the site: will it be able to adapt or do we simply end up with a husk of what HN was and it’s the AI posts driving majority of engagement, Overton window, and upvotes/downvotes?
I look forward to this, I think it is an exciting development.
Maybe "generated by claud" is a meaningless category because it encompasses both projects made via AI by people who have never used a computer before, and projects made by senior developers who used AI to do the actual coding.
Wasn't that obvious the second ChatGPT 3.5 released?
In a clip: https://youtu.be/WAZljmaRxE4?si=i1p4jn3zxgmQrKUk
The writing here is good. Quote of the day "Any fool can feed coins into a fruit machine and pull the arm."
How would one build an online community free of LLM agents commenters and links to "slop" content?
Strict invitation trees? Small signup fees? No SEO incentives?
I've always thought the "strict invitation trees" or vouch trees would be an interesting way to moderate a community, even before the LLM era. A user can vouch for an unlimited number of new accounts, but if more than 10% of the vouched accounts are banned or flagged down the line, the parent voucher acct is also banned/flagged.
Since it creates a tree structure, you can wipe out entire armies of bot/spam/otherwise accounts by following the vouches up the tree.
My guess is that sooner or later we're going to one or the other of these:
* dead online communities
* highly-invasive, government-mandated "prove you are a human" requirements in order to participate in online communities
Wait for the EU AI Act to require text watermarking in August. It will work, and it will be effective -- not because it'll be impossible to circumvent, but because all the big SaaSes will have to adopt it, and the hurdle of stripping it back out will filter out the vast majority of the sloppers.
Im not a crypto person, but I was intrigued by Chia. They generate their coins based on allocating disk space. So if you have a bit of free space, you can fill it with plots and play the lotto.
The intriguing part is that I think it works against scaling. The incremental cost for me to use the 500GB of free space on my disk is $0, but someone scaling a bot farm has to buy all their space.
Real people tend to have a lot more idle capacity than optimized, scaled businesses, so any kind of proof of idle capacity seems like it would disadvantage bot farms.
I’ve also thought that proof of collateral spending would be a good system. For example, you buy groceries and the store gives you a token saying you spent $X of real world money. Those tokens help show you're not a bot. Keeping that system honest and equitable would be extremely difficult though.
Maybe schools could give kids tokens for attendance. It sounds kind of dumb, but who knows.
The actual reality of Chia is that it drove up hard drive prices just like LLMs drove up GPU prices. People bought petabytes of space just to run Chia and if you wanted a computer you had to outbid them.
Charge $10 for an account, like Something Awful.
It'd be interesting to see how lobste.rs fares with all this.
Probably all three of those. Tildes and fediverse instances do the first, resurgence pending for the second, and lastly non-mainstream social media sites have no SEO garbage by default.
Human slop is realistically just as bad. In a strange twist, human commentary on the Internet is asymptotically approaching an older LLM. Trite cliches, repetitive tropes, and tribal affiliation signals dominate conversation.
I have turned to blunt instruments: blocking individuals on their first cliche banner-wave. It has substantially improved comment quality but I still suffer from the problem that I don’t block stories entirely.
You've written about this before so I'm curious how effective you find it? Did you try a blocklist on top of doing things like muting words? Btw I enjoy your content on this site quite a bit.
Thank you. Those are kind words! I've mostly been pretty happy about the lists I've curated blocking users. I don't use words because I'm sure I'll clbuttic/scunthorpe my way into missing something and so on. The current problem I have is that on my iPhone, I use Chrome and it doesn't have extensions so I have to view everyone. I'd much rather view people I like, so I'm going to have to make an iOS app.
Really enjoyed https://hackernews-insight.vercel.app/user-analysis
That spike in users near the end is really something!
Who's reading this comment in 2026?
First
I have been reading HN near-daily for years.
This synthetic participation (LLM or otherwise) has catalyzed weakspots in HN's high-trust environment. The weight we give to the average HN comment is orders of magnitude higher than the average Reddit (& co.) comment, and this relationship probably goes both ways (much higher ROI on ads/propaganda). Due to the low volume & high trust, it seems to be a very different (easier) environment in which to achieve pervasive propaganda/advertising/etc with a disproportionate impact.
I remember when some new LLM version came out (maybe from Meta?) I saw something like 3 of the top 10 posts on the front page were all variations of "Foobar 2.1 New Model". Perhaps not explicit, deliberate manipulation, but the result was the same, and apparently allowed. How many of those generic LLM websites (https://letsbuyspiritair.com/ comes to mind) show up on the front page per day? Zero effort static front-ends for some unremarkable data. I'm not going to touch the politics minefield, but that is a weakspot too.
All of this, and yet I think HN has handled it relatively well. I really appreciate not seeing comments of the form "I asked Clog/Gemini/etc. here's 5 paragraphs". Places like Reddit do not have the agility or control, and have degraded accordingly.
It makes me sad to think that a short time ago, every forum was ~100% humans, and now it is some fraction of that. I wonder if I will ever see that again.
AI slop is hurting my community in a different way. We have an internal viva engage community for quick development how to type questions at work. More frequently, instead of asking "how to" questions to the crowd to crowdsource answers, people are reaching out to me directly to ask me why the solution AI suggested doesn't work.
That people trust AI over an organizational knowledge is bad enough. I fear that AI is turning people generally antisocial.
This is happening at my workplace and it's incredibly annoying. We get support tickets asking us to troubleshoot AI written scripts. The funny thing is that most of the time, it would be faster for the customer to tell us what they want to do in plain english and have us make it for them. Hell, if they make an honest attempt, we can point them in the right direction and teach them.
It's frustrating because we're bundling this shitty AI with our product so we're just making more work for ourselves. Then there's the push from leadership to use more AI...
I don't think it's making people antisocial though, people just like easy solutions to their problems. We're giving them what seems like an easy solution. But it's easy for them, not easy for the reviewers.
>I fear that AI is turning people generally antisocial.
This is by design btw.
Related, from a couple of days ago: Knitting Bullshit https://katedaviesdesigns.com/2026/04/29/knitting-bullshit/
So no hope for https://xkcd.com/810/?
Good. These communities were inauthentic echo chambers for most of the past decade anyway. advertising powers "online communities." slop must be selling. Reddit died long befor gpt 1.0.
Welcome to the club
Sigh. First the article states that "coding by LLM is the way things are done right now" in 10 different ways but message boards and articles need to be protected.
We get it, the current narrative is that coding is the big thing, promoted by billionaires and scabs alike.
So, the coding narrative must be protected until the IPO of Juniper^H^H^H Anthropic happens and the whole thing implodes.
You already could have code for free and faster by using "git clone" without a company of thieves selling your own output back to you.
> AI slop is driving up the noise, and making the signal more and more difficult to discern in communities.
Thank you OP, this puts into words why I no longer look at Show HNs.
This post is slop.
There are "nice", "polite" slop enthusiasts. The ones who insist they have taste and tact. They would never post bad slop, recklessly, only the very highest-quality human-refined, curated slop. Not really slop at all, they would argue, because they gave it a careful review before posting it. They insist there's a very important difference between this premium slop and the nasty kind, and that low-quality human-authored media is actually slop, too, when you think about it. They talk about how important it is for people to use slop thoughtfully, efficiently, correctly, and that we all need to learn about and discuss slop constantly because it's the inevitable future and highly relevant for everyone.
They muddy the waters. They wheedle, rules-lawyer, carve out exceptions, and talk about how important it is to have nuance in separating virtuous applications for slop from bad ones, and that focusing on the bad ones is actually very tedious and rude. We should have polite discourse about the good things about slop and stop being so mean about bad slop, which isn't even really a problem. The bad kinds of slop will be solved soon, probably, and the harms are overstated. They colonize spaces.
If moderators don't swiftly throw these slop enthusiasts out on their ass, slightly less polite ones will post slop slightly less politely. More and more of the people participating in the space will have favorable opinions toward slop, and shout down people who object to slop. In no time at all, your community is a slop bar. Who could have imagined?
I usually type 5000 words researching for a 500 word output. It's not "write me an article on X", it's 99% my own ideas, but worded and structured and polished a bit. But I don't post them here. They are on my blog.
"It's not X, it's Y"
I'm not the arbiter on all things Godwin's Law, but either way the analogy doesn't work.
gonna start calling this effect The Slop Vanguard
I do not feel sad about lost online communities. When i was at my early 20s (early 2010s) FIDO of my university was still running and i had amazing time with some oldschool hackers there. I was too young for that community and always had a feeling that I have lost or , rather, missed something great... you know, like i was born 20 years later than i would had liked to. Now this echo conference is dead. That 486 machine was probably disconnected and thrown away somewhere. Everything dies at some point. Ask yourself : do you need the tech that gives that community vibe or do you need the people behind. I try to stick to people. As for me, i would rather have an in blood-and-flesh nerd friend instead of a whole human-driven reddit. He probably knows the answer, he is happy to help. There was an article here at HN long ago, that in average we have around 150 close contacts at a time. Some drop in, some fall out and get unconciously replaced. Going beyond that number would imply exponential increase of management costs. Those oldschool guys from FIDO, they disappeared for me without a trace. Partially because quite soon I ended up in the community of radio-engineers. Honestly, i am grateful to all people that helped me online, those who were there, who actively participated and, for some reason, cared.