I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam.
Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immediately with a cease and desist. But from the receiving end it's just because the spam is unmanageable.
I'm getting CVE fatigue with all of these super ultra critical 10/10 vulnerabilities that are some node package that compiles my frontend can get stuck if I give it a malicious regex.
It's hard to spot the stuff that actually matters.
Yep. I remember years ago seeing the website for some guy who proudly listed all the CVEs he'd discovered. Clearly he'd written some scanning tool to look at regexes in open source projects and was creating CVEs for anything that might result in exponential time execution or whatever.
This stuff has been brewing for years, but since technically you could fix all instances with minimal StackOverflow downtime [1] and a slightly different pattern, few people worked on either using engines with data structures less prone to the worst case or adding the generic workarounds for those that have them.
e.g. in cPython, until 3.11, there was no support for atomic grouping (roughly translation: "never backtrack inside of this expression"). There is little useful advice a linter can give, if there is no predictable-runtime way to express what you want within a single match step, because you really do want to unwind the stack and check for repeats (just without any of the exponential runtime stuff, please).
It's only a real issue if it is in runtime code that parses untrusted input. 99% of the regex lints/CVEs that get flagged our way are in build-time code.
I tried to raise that with my internal security team recently - don't clutter my vulnerability dashboard with issues in dev dependencies. They somewhat rightly pointed out that malware needs to be dealt even if it's a dev dependency. So my suggestion went nowhere because I guess we can't filter by type of vulnerability.
Working in the EU energy sector where we have to work with NIS2 compliance, I'd argue that your security team rightly pointed it out. I suspect that's what you mean though, and the rightly is just there because you agree with it but don't like it. We work with even more tight dependencies policies than just having alerts. We have a set of pre-approved and yearly vetted packages, like pandas or pyarrow for Python data work. Aside from that we have some isolated development environments where your pipeline can get access to something like SQLC for Go. Which is essentially where your dev dependency lives in it's own environment where it can produce the code it needs to and then submit it for approval into your regular dev environment.
Ironically we'd probably need to run Dependabot itself in a mirrored environment since it too has external dependencies we'd probably not want to vet.
I do think external dependencies are among our biggest security threats though. It's so hard to vet them, and compliance basically comes down to "We trust the apache software foundation enough, and pyarrow is vital to our business, so we accept the risks", and then you lock versions and aren't the first to update except for vulnerabilities. Shadow AI is obviously the number one security threat right now, especially in enterprise with people who are very tech savvy. This makes dependencies so much worse though, because now everyone can (if their systems aren't locked down tight) do so many crazy things. Both with the "non-sanctioned" AI but also with the code it can generate for them.
> compliance basically comes down to "We trust the [third party] enough, and [third party component] is vital to our business, so we accept the risks"
This is the beginning and end of reasonable security. This is what it's always about, and if you go beyond it, you risk practicing art for the sake of art, at the expense of customers and other stakeholders.
Security is about understanding and managing risk. Not about achieving some mathematical perfection (which is actually only achievable by making your system an inert piece of rock, but people realize that way too late, after piling on way too many pointless "security improvements").
I've almost always gotten everything I want from security teams over my career. Usually a quick and honest chat with them gets you pretty far.
I always lead with some flavor of "In my perfect world, I have 100% access and ability to everything, everywhere all the time. In your perfect world, I don't even have a computer. Here's why I need X permission / Y user group / z application"
From the perspective of big corporate security - developers are a wild nuisance who file the lions share of the tickets and soak up an inordinate amount of resources. Being able to at least explain to them that you understand their objectives and are not overrequesting just for the sake of overrequesting goes a long way.
I'm not the previous poster, but in my experience you can get a lot of mileage out of having dev teams (tediously, at least the first time) go through all potential vulnerabilities, decide how risky each one is based on likelihood and impact, and then get them to address the high/highs somehow (e.g. by upgrading a dependency, or writing extra code to guard against the issue, or fixing the issue if it's a home-grown vulnerability).
First, this is great reply with lots of real world experience to share.
> I do think external dependencies are among our biggest security threats though.
This sounds like a good business opportunity. I know that Sonatype has a business to vet Java dependencies. Does your company use it? I am guessing that Sonatype may be expanding into other open source ecosystems.
Companies like Sonatype would be an issue since they are owned by USA private equity. We would not give "Vista Equity" access to anything with the current EU US relationship. It's bad enough that we're so tied into Microsoft, which the EU might task us with leaving if they deem it critical enough for the security of the European energy sector. That's a risk we live with though, there isn't a realistic alternative.
That being said, our current strategy is more along the lines of building thind within standard libraries. We really wanted to adopt Go company wide, but it's proven impossible for non-SWE staff to use AI to create their projects in anything but Python. So instead we've created AI configurations that know our security policies, the tools we want them to use and we've setup security policies which won't even allow you to run a Python executionable inside a virtual environment unless your devices is sepcifically allowed to do so in that specific folder. Similarily we've completely limited what VSCode extensions they can use down to the named folder version. Which sort of sucks, and I doubt a lot of it would be possible if it wasn't because the c-levels are personally liable for security under EU law.
We'll see what happens after september when the summer holidays are over and the real token cost of AI will kick in.
> First, this is great reply with lots of real world experience to share.
I know how they came about with this setup, but I think that's the wrong way of approaching the problem.
Their problem is legacy and trickle-in features in an otherwise unmaintainable code.
With AI, they can rewrite their software to minimize dependencies and in general reduce the attack surface by allowing the business to automate more on their own.
But it requires bold management decisions and people in position of authority that can pick the right battles for the advancement of their careers.
Of course, all the generated code has to reviewed and vetted for by a senior developer. Of course, this has to be re-done every now and then when new classes of vulnerabilities appear that the previous generation didn’t have in mind.
Or do you just trust the AI that was trained on a lot of bogus code?
Yeah I completely understand their intent, but I might get 30 vulnerabilities across a multiple repos flagged in a week. It is already tedious to check them all and assess if they're worth worrying about let alone having to update them. These are 99% Javascript though - I suspect other ecosystems are much more manageable.
It's easier to keep stuff up to date these days. If you have a project with typescript, unit tests, and end to end tests like cypress you can just have dependabot create the PRs to update packages. If everything passes you just have to hit the merge button.
Just updating everything is probably easier than assessing if it's possible to trigger an exploit with the way you use the package.
This is exactly how developers of malware want you to behave. Update without really thinking about it.
I do wonder how long it will take before an attack is developed by submitting a semi-genuine vulnerability, shortly followed by a ‘fix’ including malicious code.
Notably this does nothing to "solve" the attack vector. You've got a live bomb in front of you and you're adding 10s to the countdown hoping that _others_* find it and defuse it in that time period.
I would challenge anyone proposing this to define more than one party doing security checks on packages to prove the point that many projects are waving their hands nebulously around saying "security vendors" and then YOLO'ing code into their codebase because they didn't here the muses wailing.
Alternatively from the other direction - Point to any dependency in your project. How can you get *POSITIVE SIGNAL* that security vendors _did_ look at it and okay it? How much scrutiny did they put into it? At what version did they last inspect it?
With today's AI glut of tokens, multiple someones are scanning security checks against the changed code. The real problem, as was before, was getting anybody anywhere to pay enough attention for long enough.
Only if their scanning detects it though. Malware authors have incentive to figure out how to fool the tool. They don't even need to be right all the time, any attack that survives works for them, and creating accounts is easy.
Dependencies and supply chain attacks are probably the greatest risk to a lot of software orgs, as they run them across all their environments: Development (with secrets and other valuable artefacts on developer VMs), CI/CD pipelines which may have access tokens to production (and other) environments, and production itself.
Notably even security companies are being impacted by this[0]. The scale of these attacks has amplified quite significantly the past three years, but are not solely exclusive to the javascript ecosystem [1] or even just namesquatting/typosquatting [2].
The resolution is broader security awareness, "onion layered" security controls and implementing simple non-burden inducing processes and policies. Sometimes not updating (what was wrong with the previous version of a dependency if there was no immediate vulnerability or production issue caused by it?) or having a two week cool down for updates (which some supply chain tooling natively supports) can appease some security functions through clear communication of the supply chain risk etc.
If anyone has interest in courses aligned to your org on improving developer and broader engineering management awareness on this, e-mails in my profile :).
Yep this is what has happened to small teams. You really only have time to approve the dependabot changes and go go go. Otherwise you'll never get anything productive done.
The other option is to simply ignore updates and do them on a schedule, e.g. once every 1-2 months.
Or you take the alternative approach of flattening and minimizing your dependency graph. Having so many dependencies you can't reasonably field bug reports in them is a chosen tradeoff, even if it doesn't feel that way.
I unironically think the solution is vibecoding your own Javascript blobs that use no frameworks and have no (or minimal) external dependencies. At this point it is entirely feasible for many kinds of projects.
Everyone talking about malware in dev dependencies as if dependabot only raises issues about that, but it does not. It raises warnings about all sort of "vulnerabilities" irrespective of the threat model.
Even worse, it incentivizes randomly updating dependencies, which is what actually allows supply chain attacks.
Dev dependencies is how they compromised SolarWinds and thereby most of the US federal government.
> The attackers used a supply chain attack. The attackers accessed the build system belonging to the software company SolarWinds, possibly via SolarWinds's Microsoft Office 365 account, which had also been compromised at some point. SolarWinds was using build management and continuous integration server TeamCity provided by the Czech company JetBrains. In 2021 The New York Times stated that unknown parties apparently embedded malware in JetBrains' software and through this way compromised also SolarWinds.
I don’t know what kind of software you write, how valuable your company’s infrastructure is, etc. But supply chain and insider threat in security/infrastructure is a big topic — that I’m sure they’re concerned about because that’s their area of responsibility.
Even if I’m personally sympathetic to not wanting to deal with the churn of dev dependency updates.
This is very real, but such CVEs are such a tiny fraction in relation to denial-of-service-due-to-regex that it’s hard to take the system seriously.
So far as I’m concerned the solution is to isolate everything as much as possible. I’d love to see something on the CVE classification side to also address the signal to noise problem but I don’t see it happening.
Pretty much - I don't know too much about the CVE process but if ReDoS stuff was flagged at the CVE level as "exploitable only with unconstrained inputs" then great - I know my tests have sane inputs, so I'll close thanks.
So I have a library and its ultimate purpose is converting globs to regexes. Someone sent me a ReDoS vulnerability report with a 4.0 CVSS score because if you write an obscene glob pattern you'll get a correspondingly obscene (and inefficient) regex. What else would you have it do!?
Vulnerable dependencies are very different to compromised or backdoored dependencies though. Noone's taking over Solarwinds because their build tools had a ReDOS involving input from their own config files.
Yeah. We used to get a bunch for OS things inside containers (the ones that had OSes in there) like "Perl regex has exploitable something-or-other". They always came in at high priority so we had to fix them, but realistically they were irrelevant.
Time to start banning those that submit fake or superficial reports. Maybe with enough bans these people will start actually reading their own vulnerabilities.
CVE 10 if you use you current version of Python to serve files over ftp, and parse the incoming files using the most obscure file type found in the forbidden libraries of the Vatican
And your ISO etc certificates make this CVE mandatory priority 1 action point
I think this one has more to do with excessive dependencies, and lack of splitting into individually installable packages and/or static linking.
I have already avoided having to evaluate whether I am affected by some issue because I added patches at startup that crash before certain unused-yet-installed modules are to be loaded.
Also, for those Python packages that still have a pure version that defers to stdlib and a separate muh-performance binary option with statically linked dependencies, I can generally just install the former and skip the version bumps for dependencies. The performance advantage may be negligible or negative outside of benchmarking 100k calls.. of code actually called 11 times a day, on a non-critical path.
CVE 10.0 if you happen to use one specific functionality in the library. Non-CVE if you do not, and because it's an old bug, you cannot just downgrade to get a non-red result from Trivy.
(besides cve, nist, et al drop in criteria) searching for an indepth analisys, you find one million (after scrolling the Ai summary) results that are either copy-pastrle or Ai rewording of the cve announcement.
...and don't get me started on the proofs that stop after smelling the attack vector. you can't evaluate if your setup is DoSable at most or full remote shell.
there's still tons of good analisys and reports. but the noise....
Same experience here. I've run a successful vulnerability disclosure program for over a decade and paid out thousands of dollars in bounties for scanii.com (a malware identification API service), but recently (since the beginning of the year), we went from receiving maybe 5 per month to receiving 5 per day. These are clearly AI-generated and extremely low quality (albeit well-written). The rules of the program aren't read, and it's clearly a “point-and-click to a website" and file a report.
I'm now considering just shutting down the program since, as the OP pointed out, if you found this vulnerability using an AI tool, they are inherently public.
I haven't gone that far yet but have instituted some new rules aiming at filtering out most of the reports: 1- No AI-generated report and 2 - Reports must include a video of the exploit.
You can see our program rules here: https://docs.scanii.com/article/131-does-scanii-have-a-secur...
What if... on the vulnerability report rules page there's an image of some text saying something like "your report must include the text: turtle123". Reports without that text get automatically deleted.
Sure - modern AI can figure that out, but I bet in a vast majority of cases they won't.
Reminds me of someone (well known in their field) who charged $0.05 for using their “contact me” page. A trivial amount for someone who genuinely wanted to contact them, but just high enough to prevent any kind of scaled abuse
If I've stumbled across what I think is a security issue in your systems, there is zero chance that I'm going to get out my credit card and pay you for the privilege of responsibly disclosing it to you. Especially if it's the vulnerability is in the site hosting the contact form.
I know some professors who have started doing something similar to combat students using AI for their work. Even going as far as to hide the "your report must include XYZ obscure word 3x" prompt instructions in small invisible text. It's gotten pretty bad, with some students turning in papers with the original ChatGPT prompt LEFT IN THE TURNED IN ASSIGNMENT.
Have you considered requiring a small payment for vulnerability disclosure? Refund it on payout. This should be very effective at deterring spammers. It also sucks for real reports, but beats shutting down the program entirely.
Those are 2 big "ifs". The incentives are completely misaligned and the platforms work for the companies. They would now have an even bigger incentive to stonewall and close valid issues than they did before.
They already like blurring the lines by rejecting reports that have clear reproduction scripts, videos, demonstrable (but not critical) impact. They'll close it as "not a bug" but then also forbid disclosure and stonewall mediation requests. Reports are supposed to be kept private until the issue is fixed but the system gets abused to cover up issues long after they've been fixed.
In some cases I strongly suspect it's to evade liability for financial damages that their customers might've suffered. Platform mediation always takes their side and if you want to do what's right, you will get banned.
It's not a horrible idea... the challenge there would be making that payment/refund flow totally transparent in order to build trust and be fair to the researchers.
Making, payment/refund setup is more complicated than „set and forget”.
First question: Do you keep money for shit reports?
Well no, you have to pay it back like credit card validation. There is no pain for posting shit report just inconvenience. There is no legal way where you can keep the money.
Because you are not providing any service not selling anything. There is no real way as a company to withhold someone’s money and that it goes through accounting.
I am not an accountant so ask some accountants why not.
To participate in the bug bounty program, you must pay ACME Inc. $1 (one U.S. dollar) per submission. This payment is non-refundable as it covers our triage costs and bounty payment processing fees. You may submit a vulnerability without paying, but you will not be eligible for receiving any bounty payments under this program.
If your disclosure otherwise meets all of the guidelines of the program, but is not eligible for a bounty, we may, in our sole discretion, award you a bounty of $1.
There is a history of companies and organisations threatening legal action against security researchers when they report vulnerabilities in their systems or products.
Sometimes even when the testing has been completely offline - I know people who have downloaded some software, carried out testing against a local copy of it, and then faced legal threats when they tried to report serious security vulnerabilities to the vendor.
It's one of the reasons that some researchers don't bother trying to talk to the vendors and just go straight to full disclosure, or if they do report to vendors they do so anonymously. But if you have to pay, that's creating a link back to yourself which makes the latter much harder.
When you report a vulnerability in a product that means you hacked the product. Hacking is illegal. If it's something that runs on your own computer you might get away but if it runs on a server then it's 100% a felony.
But do you know how many people are doing things that are even dumber right this very minute? I don't know either, but I'm sure it's larger than either of us would like to admit.
Yeah, I help review security reports for a small FOSS organization, and someone reported a "critical" vulnerability about a publicly-accessible SVN server. Like yes, that is indeed the purpose of hosting open source software. But at least that report was obviously bogus; much worse are the ones that look legitimate at first, so you have to read through dozens of AI-generated paragraphs to make sure that there's nothing valid hidden in there.
Vulnerability reports are a voluntary service to help a vendor or software project. It’s often an annoyance for the security researcher. I understand people are getting slammed and it sucks, but the main result of rejecting them is going to be an increase in full disclosure. As a note: if you have a bug (that isn’t devastating but you’d like to talk about) having an LLM write up the disclosure is a great way to check the “we disclosed responsibly and they didn’t care” box.
> I understand people are getting slammed and it sucks, but the main result of rejecting them is going to be an increase in full disclosure.
Right, what I'm saying is that letting those bugs go to full disclosure (aka being filed as public issues, like every other bug) would have been a significant damage to user safety a year ago, and it's not anymore.
I think that’s an assumption. Just because an LLM might be able to find some bugs does not mean every attacker has a packaged attack, or the right prompting. The easiest way to find a vuln is just to Google for it.
> As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam.
I don't think that is unique to the LLM era. The company I work for has been getting some form of spam vulnerability reports years before LLMs were a thing. Often similar to what you mention about 'bad CSS'.
Maybe the volume has increased a bit, but we've added in a filtering solution and I'm more distant from the reports now, so hard to be sure.
I hated these low-effort reports, so I created a simple automation that checks my security inbox, mentions me in #security on Slack for things that look legitimate so I see them quickly, and marks things that seem entirely automated as spam.
I still check the spam folder for legitimate emails, but so far there haven't been any false positives.
Its been like that for half a decade across all software. People act like finding a linux kernel bug is a big deal, completely ignoring the fact that in order to exploit that bug, the attacker has to be able to run code on your computer in the first place, which is extremely hard to do these days remotely.
Also people ironically just DGAF that much. The last actual bad exploit was log4shell in java, which given how it was introduced (i.e someone purposefully at Apache made it so a log statement can execute code, and nobody questioned it before pushing it to prod), should have been the signal for everyone to completely remove all Apache libraries from their services, but yet all the software is still being used.
If someone manages to get remote code execution at user space on a machine, the amount of damage that they can do with just that versus having a kernel level exploit is about the same.
No I agree, its a pain, but its necessary if you care about security and don't want to audit every single release for potential vulnerabilities. People don't do this, so they really don't care that much.
Security Eng here.
The whole thing is an absolute mess.
I’ve been (and still am) on both sides of the fence.
I currently have two reports (one RCE on a famous OSS ML platform, one cluster take over on a k8s related projects), both are more than 2 months old without as much as an “F you, get lost”. Just got ignored and ghosted, which hurts a lot, because I spent a lot of time finding, and verifying these (all reports with poc and patch).
BUT I understand why it’s happening, because I’m also on the receiving end.
security@ and VDPs have always received BS reports and beg-bounties, but boy oh boy, these days we have two people spending 3-4 days a week sifting through this constant flood of garbage compared to 2-3 tears ago where 1 person could triage the inbox and VDP in a day’s work max, which would’ve been considered very busy. Unfortunately we can’t just shutdown the programs or the mailbox because 1. We do occasionally get important and great stuff that actually matters, and 2. We’re a critical infra company and can’t ignore anything really.
The signal to noise ratio is almost zero, but the “what if” is keeping us swimming through this unending river of garbage and burning us out.
Overall, chaotic mess on both sides.
Ending on a doom-and-gloom note: there will be a reckoning.
(Don’t take the note too seriously though, I’m a SecEng, so I have a built-in doom multiplier lol)
I reported a fun security bug to Google recently (not high in CVSS terms but will probably make the HN front page when it goes public), and the report was auto-closed in minutes as "not reproducible". If they tried to use an LLM to reproduce it I'm not surprised, since it requires a soldering iron.
Having seen the other side of a security inbox I totally get it, and fortunately I was able to get it re-opened via backchannels. I think the future of bug reporting will run almost exclusively on reputation and connections.
AI vulnerability analysis is going to find something, it will be reported by a researcher and ignored as chaff, and then separately, later, someone will build it into an exploit and compromise a piece of critical national infrastructure
Have you considered using LLMs to perform some automated more-or-less reliable classification of incoming reports by severity, affected product, etc., then have agents try to replicate the reported findings?
Ideally the reports would also be coming in in the same structured format.
Security through obscurity was never a great strategy.. and now it’s not a strategy at all..
Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others.
This is an engineering problem. It won’t be solved by models that “only do dumb shit 1/10th as often, only 0.01% of the time now not 0.1%!” It won’t be solved by adding more models to do even more double-checking before and after the work. It won’t be solved by hoping humans catch it in review. It isn’t solvable by adding outer loops of any sort - though we may get close. To truly solve this will take serious CS research.
Almost never do software companies even attempt to design secure systems. I'm not sure this requires new fundamental research so much as slightly giving a shit.
Regulation and an ethics/licensing board à la Engineers would probably be a good start. If management knows they can’t tell you to do a bad or sloppy job because no one in your industry worth a damn will… everyone wins.
I just see unintended (but easily imaginable) consequences that don't fix anything.
Especially since the world isn't Dilbert where your boss goes "oh, authz? lol nah, just yolo it" and you go "dangit, alright boss". Instead, security requires eternal vigilance and zero missteps along the thousands a project takes in its lifetime.
I think there's a reason HNers who pitch this idea never give any concrete examples of entailments of their proposal: it doesn't even sound good superficially. e.g. How this actually changes security issues. In fact it just sounds even more convenient to blame engineers.
Yes. People don't seem to understand is that if we build our tools and libraries with models that do random dumb stuff 0.01% of the time, those bugs and leaky abstractions bubble up and grow exponentially into errors and undependable behavior.
People have been doing great research with formal methods, dependent types etc. Disciplining ourselves to truly write and understand code, using the best in math and PL theory (FP and type theory etc) is the only way forward in my opinion. We have to make correctness a value and goal, or else we will keep spinning off into psychosis and the corruption of truth.
At the end of the day security problems are just bugs like any other. If the resources to find them outweight their value then there is no sense in finding them (other than to give security researchers' jobs)
I agree they’re bugs, but the impact being far greater should justify special treatment - and it does. You’ve got bug bounties, special access and even special devices for security researchers, if you count Apple’s program where they’ll let research be done on a special iphone.
The resources to find security bugs should be weighed against the impact of them being found & exploited by bad actors, not weighed against spending those resources on other things. (And small/simple products won’t suffer as much if a problem is found, so those cases justify smaller security investments)
Rust's safety guarantees are also undecidable on an arbitrary program, but Rust still proves them for safe rust code. Why so? Because safe rust code is not an arbitrary program, it is specifically forbidden from doing some things, which becomes a pain in the ass sometimes for a coder, but the result is a subset of programs you can prove safety guarantees for.
The issue is how to design a language that restricts programmers in a way you can prove everything you care for on their programs, and still don't make life of programmers unbearably difficult, because the language allows them nothing.
Most verification is undecidable, lots of it is pspace complete. That doesn’t mean very much in practice since those are worst case bounds. People regularly solve problems that are undecidable for all practical instances that they care about.
Verifying behaviour of an arbitrary program is uncomputable. However that doesnt mean you can't have proofs of behaviour of specific programs you create.
Personally i have some doubts, a lot of research has gone into the idea without much to show for it, but its a very reasonable research area.
Part of what the research shows is that correctness-by-proof has a cost in developer effort.
If there really is a vulnerability-apocalypse due to AI, and it's not just a different flavour of AI hype, the cost of having insecure software will rise to the point that the cost of dealing with insecure or incorrect code at time of creation becomes less than the cost of ignoring it until it blows up.
I doubt it'll rise so much that we'll want to face the cost of behaviour proofs for much code at all, but it's quite possible it'll rise enough that we want to do things like prove that indices are in bounds, at compile time, so vector accesses can skip checks without compromising safety.
I fear it'll just move the problem one layer up. Sure, you've now proven that the code matches the specification - but how do you ensure the specification is watertight?
See WPA2 KRACK, you could've had a formally verified WPA2 implementation and it still would've been exploitable because the flaw was the specification itself.
It might just mean the opposite. Unergonomic and/or slow memory safe languages might not be needed anymore at some point, because LLM can check for reasonable programming patterns and can do some amount of vulnerability checking upfront. But seriously the first point -- as long as you keep to a known set of reasonable patterns (much larger set than the sets accepted by restrictive and cumbersome type systems), memory unsafe languages are actually pretty safe, and the code remains easy to understand, easy to maintain, and performant.
Already today, in my experience, widespread models like Claude 4.6 or 4.8 can quite reliably find some concurrency bugs that are easily missed by humans.
This logic enables and encourages corruption in all spheres of life. "Things are messed up as they are, so why try so hard to do the right thing?" "Let's be realistic."
It's not as black and white as you claim. 99% is obviously much better than 0%, and 99.999% would be an exceptional, unheard-of improvement. In most software projects, this would find ALL bugs.
And most "memory safe" languages are not all that memory safe. Or they need escape hooks.
EDIT: To be clear it's not AT ALL black and white. You have drunken too much of the type safety kool-aid.
I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special.
Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to minimize creating bugs in the first place. That could be new or better languages, less 3rd party dependencies, more vetted code, better linters, better fuzzers, whatever. The point the new reality of bugs being easy to find will, actually must, lead to less bugs eventually because the world can't function with easy to find bugs.
"Temporary" can be an awfully long time. There is ample evidence that discovery rate of bugs (many of which can be bucketed into vulnerabilities) in any non-trivial piece of software is more or less stable.[0] In a recent podcast episode the ex-CISO of Adobe commented that every now and then they'd take a sustained squeeze to find all occurrences of a given type of bug (ie. source of vulnerability) in a codebase. They'd find a good amount of them and fix them.
Then a year or two later they'd repeat the operation and they'd find about the same amount of same types of bugs. In many occasions in code that had been in place in the previous round and had remained essentially untouched.
Paraphrasing what the Gruqg has quipped - a large piece of software has infinity bugs. Infinity minus N is still infinity.
0: Discovery rate with regards to the time spent looking for bugs. LLM-powered bug hunting has amped up the speed with which code bases can be investigated.
Ahhh - you are talking about Adobe. I always wondered, given the never ending stream of vulnerabilities in their products, what it was about their development process that produced such appalling code in the first place.
The hope is that LLMs can scan my code every day or something like that. If I make a mistake and get it past code review, the LLM will still find it and it gets fixed right away. (better yet, make LLM an automatic reviewer on everything).
Many of the bugs we are finding in projects like curl are 20 years old - once they are fixed they are fixed and so hopefully we get all those 1-20 year old problems fixed and future scans only find new problems which is itself a big improvement in the rate. I agree that we will never reach a point where there are no bugs introduced, but we should strive to fix them faster.
I feel this sentiment is wishful thinking,but I want to start by saying I hope it turns out to be correct.
I find that often bugs will be created when using an LLM, like others have said. Saying that this can then be fixed by identifying all the bugs created by an LLM with an LLM doesnt guarantee another bug is not introduced when the LLM is addressing the initial problem.
Also, what if the LLM has a blind spot. They certainly also could be incapable of finding or fixing a bug. They dont pass any benchmark at 100% right now. Also also, guaranteeing there are no bugs in your code is like saying you have 100% test coverage, all of the tests pass, and they are written perfectly. Saying that you can simply identify and fix the bugs also assumes there is enough time and energy to find all of the bugs that exist within a project and then to address them. Even LLMs use time and energy. In a sufficiently complex system that is certainly wishful thinking.
Considering the size and complexity of a lot of modern software (like web browsers, 3d modelling software, game engines, etc.) software is just too complex to not have bugs even when created and managed by LLMs.
There will continue to be bugs in code and we will simply have to live with the fact that LLMs make it easier to exploit computer systems. I mean consider a hardware bug like Spectre [0]. If bugs like this become easier to find does that mean our existing hardware will just become obsolete more quickly? that type of problem can be addressed, but at quite a high cost.
If LLMs can trivially find bugs, then they can trivially find bugs. If they can't find any bugs that doesn't mean there are no bugs but it suggests that others can't easily find them either. So the LLMs find all the bugs problem is fixed by asking the LLMs to find them before you ship.
Read what wrote, I didn't say your program will be bug free. I said, if the LLM can trivally find the bug it will. If it can't then we're at worst, back to the state of before LLMs could find bugs, but likely much better since we fixed so many of them
So, the fact that LLMs can trivially find bugs is enough to get the bugs fixed.
You, and several others, seemed to think I was saying LLMs would fix all the bugs. I never said that. I said they'd help. Finding them is help. Writing a possible fix is often help. Writing a possible fix and seeing if they can detect a bug after the fix is applied is also help. Automating the entire things and letting LLMs fix them without review is likely not help.
The bugs I can trivially find, the bugs Claude can trivially find and the bugs Codex can trivially find are not necessarily the same. The most obvious bugs would be obvious to any of us three, but beyond that we would't agree on a definition of 'trivial bug'.
This gets worse if you factor in different harnesses built for the task, and future model updates
Sure, things will get calmer than they are now, but shipping without bugs someone else considers trivial will still require more effort than most are willing to invest
Ok, I get a better idea of what youre saying from this reply than your original comment. It wasn't helpful to me that you suggested I reread your original comment.
I agree that LLMs make finding bugs take far less time and energy. I also agree that this should mean in the long run there are less trivial to find bugs IF everyone adopts the usage of LLMs while writing and reviewing code.
It does also seem possible that LLMs are better at finding bugs than fixing them.
This is where you're wrong, I ran an experiment and told it to find bugs in a ~200 LoC project. The models are tuned in a way to where they're expected to generate issue reports so a codebase that had zero bugs, zero vulnerabilities and zero changes needed it found 3 low severity issues (cosmetic) 1 medium severity issue and 1 critical severity issue. The critical severity issue was accepting unvalidated user input, for... an echo command.
That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.
Because the behaviour of software changes over the time of development and that's how many bugs happen in the first place.
Especially if you use AI, let's say you have it implement a feature and then change your mind. In my experience AI makes as many if not more bugs than a human.
It’s not necessarily symmetrical, and in fact would be very surprising if it was. It’s a probabilistic algorithm on both sides, so the energy use to find any working program vs all bugs in a working program are fundamentally different search spaces. Not to mention the false positive rate and the human verification effort. Then even the idea of incremental security checks is potentially flawed since many security issues are non-local (ie not localized to a single module).
I think I’m on this side. I find it exceedingly unlikely that we just start producing “perfect” software all the time for everything, and at the same time start generating an order of magnitude MORE software.
> That supposes that LLMs can write secure software.
I think we're at the point that the best LLMs can indeed write software that's far more secure than your average programmer. Partly because the average is so terrible.
There's an assumption in here that every developer is spending a load of money on the latest and most capable LLMs to scan for bugs in their code before every release.
But the last couple of decades have shown us that huge numbers of developers aren't even following basic and free secure development practices, let alone pouring money into expensive scanning tools.
It may be only a matter of time before all devs remember to append ", and make no mistakes" to the end of their LLM prompts, but I don't think we as an industry will ever reach a point at which every release of every package/library/application is scanned with the most capable model available.
I mean, we've had tooling like fuzzers available for a very long time, and most devs haven't run one against their software ever, let alone before each release.
It's the human factor I think will keep this a problem essentially-forever.
LLMs are finding bugs where there aren't any and wasting human time trying to disprove the slop.
If all LLM reports were accurate, they'd be of any value. However, that's not what is happening. If you have even mentioned something about a bug bounty anywhere, waves of slop peddlers will flood you with fake reports marking every minor bug as a critical problem, hoping to catch a handful of dollars in the process.
These models do find some problems and may even provide decent suggestions to fix them (though they really want to add code above anything else, quickly leading to spaghetti if you accept it all). That's not the issue at the moment, and as long as people try to incentivize people to report bugs, the issue will remain.
I do expect this to be temporary, though. Not because LLMs will fix all the bugs, but because the flood of slop will shut down most public bug bounties.
Are you making a counterpoint that the reports are so good and must all be addresses, but the problem is "llm finding all the bugs" so fast us poor slow humana cannot keep up?
I never said that. I said they are good at helping fix them. Go read the bug reports on firefox, or Safari, or Chrome. Most of them have a fix. It might be wrong but it usually points in the right direction, which is a 1000x more than nearly all human bug reports. So, the LLM helps. which is all I stated.
At risk of quoting too much of the article, it opens with this:
> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all.
> Except…
> For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t apply to vulnerability reports. No, vulnerability reports are special. Security researchers are doing us a favor by reporting things confidentially instead of doing full disclosure, so we owe them something, which is not true of regular issues opened on the issue tracker.
[...]
> It’s 2026 and none of the premises are true anymore.
I respectfully disagree.
The premise is absolutely still true: if someone discovers a critical, exploitable vulnerability in your software, the impact and tradeoffs are exactly the same as they were before LLMs started finding bugs. There are just more of them now, so they're easier to come by.
But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
All of that said, I don't think finding vulnerabilities has really been the difficult security problem for most companies (or open source projects). The difficult problem is dedicating resources to fixing those vulnerabilities instead of building software, products, and/or infrastructure that people want. That problem is absolutely still here today, but I'm optimistic that agentic security developers will be able to take the burden off of development teams in the near future.
The problem is there used to be a fairly high correlation between ‘security report’ and ‘real vulnerability’. Not perfect but good enough. Now the two are almost entirely disconnected.
> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
I think your logic is partly correct but the fact that the same LLMs are allowing an exponential increase in insecure code generated is a counterbalancing point. I do not think this phenomena will slow down.
Nah, those same LLMs, if prompted correctly, will be able to do an audit pass and a fix pass on that LLM-generated code. It’s a tooling issue that will get fixed in time.
> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report.
That is not my experience at all. People will continue to high-volume spam intended behaviour as if it is a bug.
There will be fewer reports that matter as you fix things - but the volume of reports will either stay steady or go up. Making it harder to even notice the ones that matter.
The problem always existed, but nobody amassed a sufficiently large army of trolls to exploit it until now. So it wasn't a priority to solve it before, but now it is. We're going to have to learn to differentiate reports that matter from those that don't. Classifying reports might actually be something you could productively use an LLM for..
When we solved this problem for email... We just dumped everything similar to untrusted into the rubbish bin. Important things can vanish too, and that's an acceptable price.
Yeah unfortunately there will be some high quality reports that get erroneously discarded, but so long as the signal/noise ratio improves sufficiently that's an acceptable price.
LLMs find more vulnerabilities than people because people time is heaps more expensive than LLM time, that’s it.
We’ve always been able to find heaps, we’ve just never had the right structures to put in the effort and renumerate people for looking (even if they don’t find anything).
There are some problems with incentives in the vuln report space. People report trivial vulns and expect the same treatment as people reporting critical vulns. But this isn't new with AI. Look at all the ReDos vulns in npm ecosystem. Its questionable if its a vuln in general but half of them aren't even triggerable.
They weren't special even before LLMs. Drive-by script-kiddies would run some basic scripts against your platform and send generally-not-actually-a-vulnerability reports, claiming that these were big problems, and requesting to be paid bug bounties.
And then they submit them to a CNA and get a CVE assigned, and then _everyone_ needs to deal with the not-actually-a-vulnerability report, especially when the not-actually-triggerable-DOS gets assigned a "Critical" CVSS score from EUVD or NVD.
I agree with this. One of the consequences of the "vulnpocalpyse" is that it's become even harder to sift through the noise: I triage well over a dozen reports a week, many of which are "real" in the sense that they reflect a genuine defect but otherwise have an unclear impact on a typical user. This has always been true of the median vulnerability report, but the volume means that I now lean much more heavily away from coordinated disclosure.
One flipside to this is that, because many of these bugs are "shallow" to LLMs, it's actually easier than ever to moderate the worst participants in your vulnerability program -- if someone sends you slop, you can just ban them and wait for the next, better orchestrated LLM to send you a better report for the same vulnerability.
I read every piece like this one as: Money is moving in the vulnerability space now, when as before the LLM hype incentivized that, your best bet was that someone skilled enough would accept living with the financial insecurity of being a gig worker to hopefully stumble upon your projects bug bounty program.
Is the bet here is that the hype lasts, and that people willingly will keeping on paying Dario to be able to contribute?
> But give it 1-3 months and the open models will catch up.
I wish that this would stopped being thrown around, what is this timeline based on? How good is your open model from between March and May?
Also, having read "Gödel, Escher, Bach" I know that the hare never catches up with the turtle.
The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now.
Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstration, the communication that was special.
When you realize that this is being written from the perspective of someone who does vulnerability reporting in a professional capacity, you’ll connect the dots. We took care to be kind and succinct because for many of us, we learned our skills from being on the development side of things first.
Vulnerability reports aren’t special anymore. The only ones that felt special were the ones with human touch, the ones doing their job as an adversarial thinker, and taking the care to understand that net positive outcomes require coordination even if both parties don’t see eye to eye.
Nothing has changed. It never was. You’re just inundated with AI slop; which as a practitioner who uses AI regularly I can say with absolute confidence. The end result is the same, the volume is increased, but the special thing was never the report itself.
Finding a vulnerability was always the easy but high toil part. It was the care to communicate succinctly and be invested in the outcome that was special.
I’ve been screaming this from the rooftops. Impact is what was always important. No one is going to take down prod to do an emergency patch on an RCE that COULD NEVER ACTUALLY BE EXPLOITED.
I feel like we’re witnessing the result of multiple roles suddenly becoming security aware but not having the background or understanding to make any sense of it.
Yep, I've updated dependencies with an RCE that can't be exploited in my codebase just to keep my security team happy. Not worth the multiple arguments about it not actually being an issue.
You can never guarantee that the codepath of a dependency that is vulnerable can not be reached or used as a gadget in an exploit chain. Patching dependencies, even when no direct vulnerability arises is an essential part of defense in depth and sevurity hygene.
I don't disagree, though I should have been more specific - I updated dependencies at extremely short notice because that's what security wanted. I would have updated the dependencies sooner or later anyway because I know that periodically updating dependencies is generally a good thing to do.
You can also never guarantee the patched software doesn't include a worse vulnerability, I would submit that patching software without proper time to validate changes is also a security issue.
If you aren't careful, that is how you get this security theater.
I read a casual teams message from one consultant type of guy who said "the low level linux hackers who don't care about best practices, code quality and security get what they deserve". I don't know how to feel about this other than disgust. The entire space has been overrun by LLMs making stuff up or finding things that aren't critical while some roles capitalize on the fear. There is just so much wrong with saying such things in such a way, I couldn't describe.
This entire demeanor of the way people and ai people talk about software and technical people is getting absurd and simply unprofessional. It is as if we stopped being adults.
Cannot agree more. As part of our SOC2 we have to log and respond to all inbound vulnerability reports. Before it was easy to tell if a report was just a bounty hunter looking for a low hanging fruit. Now well crafted emails with seemingly legitimate disclosures take a lot of time to validate and triage.
Our solution was to build a tool that uses LLMs to assess the report before it gets to us. Honestly I wish we didn’t have to do this but it works and has really allowed us to spend our time on the actual good reports. (Feel free to check it out at fortworx.com if you want)
> The security researchers are not special, the insight and confidentiality are
vs
> The bottleneck now is not finding potential issues but assessing which ones are real. Unless there’s already a trust relationship, external researchers can’t meaningfully contribute
My take-away from this is that the researchers were special all along and you should probably be building a trust relationship with them.
Despite what I want to believe about tech being a meritocracy, the reality is that trust plays an extremely important role and without it we risk a collapse of our open source software ecosystem.
One of my biggest criticisms of AI is the trust vacuum within which it operates
Can anybody say what is going to happen? It's not a rhetorical question, and the implied or entailed context might involve a Nash equilibrium of some sort.
Right now the rate of signal is high, and the ratio of noise is proportionally high. But it seems like everyone expects the signal to eventually plateau or sharply decline. Almost as if there is a finite supply of "low hanging fruit" for shallow scanning machines to easily discover, and then there will be some kind of new world that follows where only truly difficult problems emerge.
But eventually the question then becomes why even bother with Rust or any other silly borrow checking ideas if we can use more enjoyable programming languages with LLM side-kicks to catch security vulnerabilities on the front side of the development workflow?
IT seems to me if we exhaust all the extant security vulnerabilities to a calculus that asymptotically goes to infinitesimal zeroness, then... the only trick remaining is to scan code before it becomes vulnerable.
I'm wondering whether this is a permanent change. After all the easy-to-find bugs have fixed and you can't find them just by asking an AI, perhaps security issues will deserve special treatment again.
I wonder if LLM's 'jagged intelligence' will come to bite here again. AI might be better at finding certain kind of bugs, to the degree outperfoming the best humans, but that doesn't mean a skilled human can't find an issue which is hard for LLMs incredibly easily.
The inbound stuff we get through the vulnerability e-mail is pretty much exclusively spam. Then they start e-mailing random people of the company they can find to get through, and in the end it's still spam.
Linus Torvalds once went on record saying security vulnerabilities are no more important than regular bugs.
This of course made vulnerability researchers seethe worse than aggrieved Redditors.
It turns out he was right all along.
The author also gets it wrong by assuming that regular bug reporters are not "providing a service". They are.
When I wrote up a bug report, I made sure it's thorough with detailed steps to reproduce. It takes a lot of time and I've done it professionally for projects you've absolutely heard of.
Having said that, getting them ignored repeatedly and — even worse — having my detailed PRs rejected, sometimes within minutes, as if I'm some ignorant luser is why I don't do it anymore. My time is more valuable than your hubris.
A lot of open source developers have their heads so far up their own asses they forgot that it takes a community for projects to be successful.
> Linus Torvalds once went on record saying security vulnerabilities are no more important than regular bugs
Linus also regularly calls the contributions to Linux kernel "garbage". These contributors, btw, are the reason that autist doesn't have to work for a living, where he'd find out what the real world is like
In the end, sorting prs and vulnerabilities has been the same for open source maintainers. How about adding a credibility score to every github account ? Couldn’t that cut sorting times ?
This whole blog post makes me sad. I've been active on both sides of the vulnerability disclosure process for well over a decade and have reported a whole bunch [1] of security bugs to the Go security team. I was there back when Filippo was running the show and have continued since Roland took over. My experience with the people there has always been great.
> Ultimately, it all stems from our responsibility to our users. The security researchers are not special, the insight and confidentiality are, and we need them to keep our users safe. Ignoring a security report communicates you don’t care about users’ security, and it’s rightly a reason for shame.
100%. This was always true and I still think it is. LLMs don't change anything. At most they shift the balance and force a temporary compromise.
> LLMs are as good as almost any security researcher
This statement is extremely dependent on the definition of a security researcher. It might hold if you consider anyone with a HackerOne account, but if you restrict the definition to people who actually put in some effort, it's just not true. LLMs can find some real vulnerabilities, yes, but they also spew unprecedented volumes of garbage that an expert can immediately recognize as such.
> The insight is not scarce and precious anymore. The bottleneck now is not finding potential issues but assessing which ones are real.
Assessing which ones are real should be part of the insight. Real researchers will not submit 150 pages of spam, and three real bugs hidden in 150 pages of spam are not insight. In most cases a researcher will spend significant effort on triage before submitting anything, and an LLM still cannot do that reliably.
> Confidentiality, embargoes, and coordination also don’t matter nearly as much as they used to.
I'd argue these now matter more: the one thing LLMs do seem to do fairly well is figure out specific things based on sufficient information and a scope that's limited enough. So a plain commit containing a security fix is now much easier and cheaper to turn into an exploit than it was before.
> The years of vulnerability reports being special might be over, as weird and uncomfortable as that feels.
I'd hope not. Bug bounties might be over unless someone can figure out the spam problem, but disclosure programs that don't offer monetary incentives are probably just going through a tough period that will eventually calm down as the operators of LLMs realize the costs and do the math.
Unreliable reports have always been an issue and will remain one, LLMs or no. When it gets worse, like in the current influx of LLM-generated reports, the focus should be on identifying reliable researchers, building relationships, and providing guidance on how they can make the reports easier to triage.
Researchers are not special, but the insight they can provide totally is. LLMs might force everyone to make better use of that insight, instead of just consuming bug reports and drowning in triage.
Thanks for the comment, I was actually hoping to get your take on this! I linked to it from the article.
> Still on Hacker News, Juho Forsén, one of the most prolific reporters of Go security issues, wrote a long interesting comment that makes the argument that instead we should lean harder into trust relationships with individual researchers. It'd certainly be worth it with Juho, in retrospect, but it's unclear if it would pay off often enough, in the same way that training new contributors who might leave the project in a month or two is not always worth it.
I've run a disclosure program for ~7 years, which is an open paid program. However, over that time we've developed relationships with the most active and successful contributors, to the point that we'll now give them early access to new features to try out (all still paid for on the basis of rewards for problems found). This is proving especially valuable now in triaging the new deluge of noise from impactful issues.
> We're keeping our vulnerability disclosure program open - because even though they are rare, the genuine critical reports we receive, in amongst the noise, are still highly valuable. I don't think we're at the stage yet where finding those issues is a purely mechanical process; persistent, imaginative researchers still make a contribution to the process by finding things that LLMs by themselves, so far, haven't.
I found a DoS vulnerability in Coinbase several months ago on Hacker One. It took me literally 30 minutes to find. First time I did this in my life. I could craft a message cheaply which, when sent as the HTTP payload to a specific endpoint, would cause the server to hang for a full 30 or so seconds before getting a response. I could have easily scaled up that attack, cheaply...
I filed a report, they marked it as 'informative' and thanked me, recommended I keep looking for more vulnerabilities, but no payment at all; they said I had to be able to demonstrate major disruption of service... Which I presume is illegal. I literally showed them all the ingredients of the attack, the exact curl commands, payloads, the exact response delay could be easily be verified; you could see the server response slowing down proportional to the degree of nesting in the payload. I could execute it without authentication too; so it was essentially certain that the attack could be scaled but they made it impossible to get a reward.
The hardest part was writing the report which took several hours.
So yeah, 30 minutes of looking for a vulnerability, no prior experience in security research, first project I looked into on Hacker One, ever... A company in crypto sector which is a major target of hackers and takes security relatively seriously.
Imagine how insecure most software is! Imagine how bad most vibe-coded software is especially! Companies might as well run their servers directly inside Kim Jong Un's data center in North Korea.
North Korean hackers probably have a dashboard which shows more detailed and accurate platform analytics than what the founders of the company can see.
I think the main impact of this is that a successful career in vulnerability research is going to require a high level of proficiency in exploit development, as that's where the demonstrable real-world impact lies.
Honestly, even that is something LLMs are becoming scarily good at. Not all of it, but they have surprised me, including in terms of exploit development, more than I am comfortable with.
These kinds of posts act like there wasn't already a rich suite of automated security scanning tools available. The fact you can put them in a LLM's hand or point it at a codebase to do the exact same thing isn't really a big leap, the spam is. It's a bunch of amateurs thinking they can do work outside of the realm of their expertise now, and this trend isn't unique to security, you see it spreading across the various realms of expertise out there in the world. The world's slowly filling with slop, and CVE spam is a byproduct of that.
but like, if you mean literally "someone gave them money and they played a game of chess", the number becomes much bigger. Chess coaches, streamers, club instructors, exhibition players, league players, camp counselors, and titled players receiving appearance fees, etc. All told, you're looking at ten's of thousands across the world.
"LLMs are as good as almost any security researcher"
Oh really? If LLMs were as good as almost any security researcher then you wouldn't be getting flooded by bullshit reports from them. You'd be receiving legitimate reports instead.
Yet again it's <person who is not an X> saying that AI can replace the job of an X. Take a hint people, if AI can't replace your job it can't replace anyone elses'.
>> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation.
I don't think the gift analogy works well. In most cultures, turning down or even ignoring a gift is considered anywhere from impolite to hugely offensive. But that's the opposite of open source: there's nothing wrong with requesting changes to a PR or even closing it.
> If a security vulnerability is reported by someone who is also violating the CoC, what do you do? Do you ignore it? Fix it silently?
Is this even a question? You triage and fix the vulnerability just like any other one. Are truths spoken by folks one dislikes — even for perfectly valid reasons — any less true?
The only way I can imagine this somehow applying is if someone has a habit of reporting vulnerabilities which do not exist, or of exaggerating their severity. Is crying wolf a CoC violation? If so, then I can imagine that particular sort of bad behaviour justifying some consideration before acting on a report.
really? From what I have seen so far most of the contributions to xlibre have been cleaning up stuff thats been obviously wrong in xorg, and using tools - including AI and more bog standard tooling to find issues, add safety checks, and a whole lot more. They're merging in a namespace extension which solves many of the purported 'security' issues in X11 today.
How badly are they violating the code of conduct? It wouldn't be the first time a security researcher got thrown into prison or jail, in this line of work.
> LLMs are as good as almost any security researcher, and anyone4 can run them.
What is this, rage bait? It's bullshit, and insulting to actual security researchers.
That might be true for low-effort vulnerabilities and fake security researchers, but the real security researchers are far from being replaced by LLMs.
I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam.
Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immediately with a cease and desist. But from the receiving end it's just because the spam is unmanageable.
I'm getting CVE fatigue with all of these super ultra critical 10/10 vulnerabilities that are some node package that compiles my frontend can get stuck if I give it a malicious regex.
It's hard to spot the stuff that actually matters.
Yep. I remember years ago seeing the website for some guy who proudly listed all the CVEs he'd discovered. Clearly he'd written some scanning tool to look at regexes in open source projects and was creating CVEs for anything that might result in exponential time execution or whatever.
It sounds like an interesting case study. Do these things get reported with a patch?:
(a) add a new function that does regular expressions searching / matching with a resource checker (eg a timer);
(b) write a local linter that reports an error for any use of the builtin regular expression tools;
(c) fix all the lint warnings;
(d) commit the linter.
This stuff has been brewing for years, but since technically you could fix all instances with minimal StackOverflow downtime [1] and a slightly different pattern, few people worked on either using engines with data structures less prone to the worst case or adding the generic workarounds for those that have them.
e.g. in cPython, until 3.11, there was no support for atomic grouping (roughly translation: "never backtrack inside of this expression"). There is little useful advice a linter can give, if there is no predictable-runtime way to express what you want within a single match step, because you really do want to unwind the stack and check for repeats (just without any of the exponential runtime stuff, please).
[1]: https://meta.stackoverflow.com/questions/328376/why-does-sta...
No I think he was just looking to raise his profile, not to help.
already exists: https://joshua.hu/comparing-redos-detection-tools. `recheck` and `redos-detector` are definitely the best. there's even an eslint plugin for the latter: https://github.com/tjenkinson/eslint-plugin-redos-detector
That's a real issue, took cloudflare down once...
It's only a real issue if it is in runtime code that parses untrusted input. 99% of the regex lints/CVEs that get flagged our way are in build-time code.
Seriously. We got 116 github dependabot alerts this week. Half of them for dev dependencies.
I got reminded every week that my static site generator "Jekyll" is insecure.
Ok. Hacking me by changing the input to my Jekyll rather involves being on the other side of the airtight hatch.
I tried to raise that with my internal security team recently - don't clutter my vulnerability dashboard with issues in dev dependencies. They somewhat rightly pointed out that malware needs to be dealt even if it's a dev dependency. So my suggestion went nowhere because I guess we can't filter by type of vulnerability.
Working in the EU energy sector where we have to work with NIS2 compliance, I'd argue that your security team rightly pointed it out. I suspect that's what you mean though, and the rightly is just there because you agree with it but don't like it. We work with even more tight dependencies policies than just having alerts. We have a set of pre-approved and yearly vetted packages, like pandas or pyarrow for Python data work. Aside from that we have some isolated development environments where your pipeline can get access to something like SQLC for Go. Which is essentially where your dev dependency lives in it's own environment where it can produce the code it needs to and then submit it for approval into your regular dev environment.
Ironically we'd probably need to run Dependabot itself in a mirrored environment since it too has external dependencies we'd probably not want to vet.
I do think external dependencies are among our biggest security threats though. It's so hard to vet them, and compliance basically comes down to "We trust the apache software foundation enough, and pyarrow is vital to our business, so we accept the risks", and then you lock versions and aren't the first to update except for vulnerabilities. Shadow AI is obviously the number one security threat right now, especially in enterprise with people who are very tech savvy. This makes dependencies so much worse though, because now everyone can (if their systems aren't locked down tight) do so many crazy things. Both with the "non-sanctioned" AI but also with the code it can generate for them.
> compliance basically comes down to "We trust the [third party] enough, and [third party component] is vital to our business, so we accept the risks"
This is the beginning and end of reasonable security. This is what it's always about, and if you go beyond it, you risk practicing art for the sake of art, at the expense of customers and other stakeholders.
Security is about understanding and managing risk. Not about achieving some mathematical perfection (which is actually only achievable by making your system an inert piece of rock, but people realize that way too late, after piling on way too many pointless "security improvements").
I've almost always gotten everything I want from security teams over my career. Usually a quick and honest chat with them gets you pretty far. I always lead with some flavor of "In my perfect world, I have 100% access and ability to everything, everywhere all the time. In your perfect world, I don't even have a computer. Here's why I need X permission / Y user group / z application"
From the perspective of big corporate security - developers are a wild nuisance who file the lions share of the tickets and soak up an inordinate amount of resources. Being able to at least explain to them that you understand their objectives and are not overrequesting just for the sake of overrequesting goes a long way.
Asking out of curiousity - how would you or how does your org handles this right now?
I'm not the previous poster, but in my experience you can get a lot of mileage out of having dev teams (tediously, at least the first time) go through all potential vulnerabilities, decide how risky each one is based on likelihood and impact, and then get them to address the high/highs somehow (e.g. by upgrading a dependency, or writing extra code to guard against the issue, or fixing the issue if it's a home-grown vulnerability).
First, this is great reply with lots of real world experience to share.
This sounds like a good business opportunity. I know that Sonatype has a business to vet Java dependencies. Does your company use it? I am guessing that Sonatype may be expanding into other open source ecosystems.Companies like Sonatype would be an issue since they are owned by USA private equity. We would not give "Vista Equity" access to anything with the current EU US relationship. It's bad enough that we're so tied into Microsoft, which the EU might task us with leaving if they deem it critical enough for the security of the European energy sector. That's a risk we live with though, there isn't a realistic alternative.
That being said, our current strategy is more along the lines of building thind within standard libraries. We really wanted to adopt Go company wide, but it's proven impossible for non-SWE staff to use AI to create their projects in anything but Python. So instead we've created AI configurations that know our security policies, the tools we want them to use and we've setup security policies which won't even allow you to run a Python executionable inside a virtual environment unless your devices is sepcifically allowed to do so in that specific folder. Similarily we've completely limited what VSCode extensions they can use down to the named folder version. Which sort of sucks, and I doubt a lot of it would be possible if it wasn't because the c-levels are personally liable for security under EU law.
We'll see what happens after september when the summer holidays are over and the real token cost of AI will kick in.
> First, this is great reply with lots of real world experience to share.
I know how they came about with this setup, but I think that's the wrong way of approaching the problem.
Their problem is legacy and trickle-in features in an otherwise unmaintainable code.
With AI, they can rewrite their software to minimize dependencies and in general reduce the attack surface by allowing the business to automate more on their own.
But it requires bold management decisions and people in position of authority that can pick the right battles for the advancement of their careers.
Of course, all the generated code has to reviewed and vetted for by a senior developer. Of course, this has to be re-done every now and then when new classes of vulnerabilities appear that the previous generation didn’t have in mind.
Or do you just trust the AI that was trained on a lot of bogus code?
The latter, of course.
Yeah I completely understand their intent, but I might get 30 vulnerabilities across a multiple repos flagged in a week. It is already tedious to check them all and assess if they're worth worrying about let alone having to update them. These are 99% Javascript though - I suspect other ecosystems are much more manageable.
It's easier to keep stuff up to date these days. If you have a project with typescript, unit tests, and end to end tests like cypress you can just have dependabot create the PRs to update packages. If everything passes you just have to hit the merge button.
Just updating everything is probably easier than assessing if it's possible to trigger an exploit with the way you use the package.
This is exactly how developers of malware want you to behave. Update without really thinking about it.
I do wonder how long it will take before an attack is developed by submitting a semi-genuine vulnerability, shortly followed by a ‘fix’ including malicious code.
Dependency cooldowns fix most of those problems.
The cooldown setting in dependabot solves this attack vector. By setting it you give security vendors time to scan new packages.
Notably this does nothing to "solve" the attack vector. You've got a live bomb in front of you and you're adding 10s to the countdown hoping that _others_* find it and defuse it in that time period.
I would challenge anyone proposing this to define more than one party doing security checks on packages to prove the point that many projects are waving their hands nebulously around saying "security vendors" and then YOLO'ing code into their codebase because they didn't here the muses wailing.
Alternatively from the other direction - Point to any dependency in your project. How can you get *POSITIVE SIGNAL* that security vendors _did_ look at it and okay it? How much scrutiny did they put into it? At what version did they last inspect it?
With today's AI glut of tokens, multiple someones are scanning security checks against the changed code. The real problem, as was before, was getting anybody anywhere to pay enough attention for long enough.
Only if their scanning detects it though. Malware authors have incentive to figure out how to fool the tool. They don't even need to be right all the time, any attack that survives works for them, and creating accounts is easy.
In agreement with frodd above.
Dependencies and supply chain attacks are probably the greatest risk to a lot of software orgs, as they run them across all their environments: Development (with secrets and other valuable artefacts on developer VMs), CI/CD pipelines which may have access tokens to production (and other) environments, and production itself.
Notably even security companies are being impacted by this[0]. The scale of these attacks has amplified quite significantly the past three years, but are not solely exclusive to the javascript ecosystem [1] or even just namesquatting/typosquatting [2].
The resolution is broader security awareness, "onion layered" security controls and implementing simple non-burden inducing processes and policies. Sometimes not updating (what was wrong with the previous version of a dependency if there was no immediate vulnerability or production issue caused by it?) or having a two week cool down for updates (which some supply chain tooling natively supports) can appease some security functions through clear communication of the supply chain risk etc.
If anyone has interest in courses aligned to your org on improving developer and broader engineering management awareness on this, e-mails in my profile :).
[0] - https://socket.dev/blog/ongoing-supply-chain-attack-targets-...
[1] - https://orca.security/resources/blog/hades-pypi-supply-chain...
[2] - https://checkmarx.com/zero-post/python-pypi-supply-chain-att...
Yep this is what has happened to small teams. You really only have time to approve the dependabot changes and go go go. Otherwise you'll never get anything productive done.
The other option is to simply ignore updates and do them on a schedule, e.g. once every 1-2 months.
Or you take the alternative approach of flattening and minimizing your dependency graph. Having so many dependencies you can't reasonably field bug reports in them is a chosen tradeoff, even if it doesn't feel that way.
I unironically think the solution is vibecoding your own Javascript blobs that use no frameworks and have no (or minimal) external dependencies. At this point it is entirely feasible for many kinds of projects.
Sure, so you'll be vulnerable to all sort of things because vibe code is really insecure (as I've personally seen).
Everyone talking about malware in dev dependencies as if dependabot only raises issues about that, but it does not. It raises warnings about all sort of "vulnerabilities" irrespective of the threat model.
Even worse, it incentivizes randomly updating dependencies, which is what actually allows supply chain attacks.
A lot of the recent npm attacks have been exfiltration from dev machines, which would just as likely from dev dependencies.
Developer's machines and cicd systems are high value targets. They were absolutely right to point that out.
Only in the castle and moat security model popularized by Microsoft and the various "security" vendors that leech off it.
And the money wasted on the security theatre around this outdated concept is astonishing.
Dev dependencies is how they compromised SolarWinds and thereby most of the US federal government.
> The attackers used a supply chain attack. The attackers accessed the build system belonging to the software company SolarWinds, possibly via SolarWinds's Microsoft Office 365 account, which had also been compromised at some point. SolarWinds was using build management and continuous integration server TeamCity provided by the Czech company JetBrains. In 2021 The New York Times stated that unknown parties apparently embedded malware in JetBrains' software and through this way compromised also SolarWinds.
https://en.wikipedia.org/wiki/2020_United_States_federal_gov...
I don’t know what kind of software you write, how valuable your company’s infrastructure is, etc. But supply chain and insider threat in security/infrastructure is a big topic — that I’m sure they’re concerned about because that’s their area of responsibility.
Even if I’m personally sympathetic to not wanting to deal with the churn of dev dependency updates.
This is very real, but such CVEs are such a tiny fraction in relation to denial-of-service-due-to-regex that it’s hard to take the system seriously.
So far as I’m concerned the solution is to isolate everything as much as possible. I’d love to see something on the CVE classification side to also address the signal to noise problem but I don’t see it happening.
Pretty much - I don't know too much about the CVE process but if ReDoS stuff was flagged at the CVE level as "exploitable only with unconstrained inputs" then great - I know my tests have sane inputs, so I'll close thanks.
So I have a library and its ultimate purpose is converting globs to regexes. Someone sent me a ReDoS vulnerability report with a 4.0 CVSS score because if you write an obscene glob pattern you'll get a correspondingly obscene (and inefficient) regex. What else would you have it do!?
These DoS Regex 10/10 CVEs in some minor helper function in some package that is used once in some random side code pathway are so damn annoying.
If I could filter out DoS CVEs‚ I would.
Vulnerable dependencies are very different to compromised or backdoored dependencies though. Noone's taking over Solarwinds because their build tools had a ReDOS involving input from their own config files.
It's even worse if you're SOC2 because then you actually have to go through and mark them as "not exploitable." The noise is insane right now.
Yeah. We used to get a bunch for OS things inside containers (the ones that had OSes in there) like "Perl regex has exploitable something-or-other". They always came in at high priority so we had to fix them, but realistically they were irrelevant.
Time to start banning those that submit fake or superficial reports. Maybe with enough bans these people will start actually reading their own vulnerabilities.
CVE 10 if you use you current version of Python to serve files over ftp, and parse the incoming files using the most obscure file type found in the forbidden libraries of the Vatican
And your ISO etc certificates make this CVE mandatory priority 1 action point
I think this one has more to do with excessive dependencies, and lack of splitting into individually installable packages and/or static linking.
I have already avoided having to evaluate whether I am affected by some issue because I added patches at startup that crash before certain unused-yet-installed modules are to be loaded. Also, for those Python packages that still have a pure version that defers to stdlib and a separate muh-performance binary option with statically linked dependencies, I can generally just install the former and skip the version bumps for dependencies. The performance advantage may be negligible or negative outside of benchmarking 100k calls.. of code actually called 11 times a day, on a non-critical path.
CVE 10.0 if you happen to use one specific functionality in the library. Non-CVE if you do not, and because it's an old bug, you cannot just downgrade to get a non-red result from Trivy.
Not sure what dumbass out there is marking those as 10/10. A 10 should be an auth bypass or RCE. Not a crashed build in my CI.
The common thread of late really seems to be the node ecosystem
we killed the curators.
(besides cve, nist, et al drop in criteria) searching for an indepth analisys, you find one million (after scrolling the Ai summary) results that are either copy-pastrle or Ai rewording of the cve announcement.
...and don't get me started on the proofs that stop after smelling the attack vector. you can't evaluate if your setup is DoSable at most or full remote shell.
there's still tons of good analisys and reports. but the noise....
Same experience here. I've run a successful vulnerability disclosure program for over a decade and paid out thousands of dollars in bounties for scanii.com (a malware identification API service), but recently (since the beginning of the year), we went from receiving maybe 5 per month to receiving 5 per day. These are clearly AI-generated and extremely low quality (albeit well-written). The rules of the program aren't read, and it's clearly a “point-and-click to a website" and file a report. I'm now considering just shutting down the program since, as the OP pointed out, if you found this vulnerability using an AI tool, they are inherently public. I haven't gone that far yet but have instituted some new rules aiming at filtering out most of the reports: 1- No AI-generated report and 2 - Reports must include a video of the exploit. You can see our program rules here: https://docs.scanii.com/article/131-does-scanii-have-a-secur...
What if... on the vulnerability report rules page there's an image of some text saying something like "your report must include the text: turtle123". Reports without that text get automatically deleted.
Sure - modern AI can figure that out, but I bet in a vast majority of cases they won't.
Reminds me of someone (well known in their field) who charged $0.05 for using their “contact me” page. A trivial amount for someone who genuinely wanted to contact them, but just high enough to prevent any kind of scaled abuse
If I've stumbled across what I think is a security issue in your systems, there is zero chance that I'm going to get out my credit card and pay you for the privilege of responsibly disclosing it to you. Especially if it's the vulnerability is in the site hosting the contact form.
That actually great idea. What payment method or processor used?
I know some professors who have started doing something similar to combat students using AI for their work. Even going as far as to hide the "your report must include XYZ obscure word 3x" prompt instructions in small invisible text. It's gotten pretty bad, with some students turning in papers with the original ChatGPT prompt LEFT IN THE TURNED IN ASSIGNMENT.
Have you considered requiring a small payment for vulnerability disclosure? Refund it on payout. This should be very effective at deterring spammers. It also sucks for real reports, but beats shutting down the program entirely.
Why would anyone pay money to have a chance of being arrested?
If a vulnerability disclosure program has a good track record of paying out, and legitimate reports get refunded, why not?
Again, the alternative might be shutting down the program entirely.
Those are 2 big "ifs". The incentives are completely misaligned and the platforms work for the companies. They would now have an even bigger incentive to stonewall and close valid issues than they did before.
They already like blurring the lines by rejecting reports that have clear reproduction scripts, videos, demonstrable (but not critical) impact. They'll close it as "not a bug" but then also forbid disclosure and stonewall mediation requests. Reports are supposed to be kept private until the issue is fixed but the system gets abused to cover up issues long after they've been fixed.
In some cases I strongly suspect it's to evade liability for financial damages that their customers might've suffered. Platform mediation always takes their side and if you want to do what's right, you will get banned.
It's not a horrible idea... the challenge there would be making that payment/refund flow totally transparent in order to build trust and be fair to the researchers.
Making, payment/refund setup is more complicated than „set and forget”.
First question: Do you keep money for shit reports?
Well no, you have to pay it back like credit card validation. There is no pain for posting shit report just inconvenience. There is no legal way where you can keep the money.
Why not?
Because you are not providing any service not selling anything. There is no real way as a company to withhold someone’s money and that it goes through accounting.
I am not an accountant so ask some accountants why not.
To participate in the bug bounty program, you must pay ACME Inc. $1 (one U.S. dollar) per submission. This payment is non-refundable as it covers our triage costs and bounty payment processing fees. You may submit a vulnerability without paying, but you will not be eligible for receiving any bounty payments under this program.
If your disclosure otherwise meets all of the guidelines of the program, but is not eligible for a bounty, we may, in our sole discretion, award you a bounty of $1.
it's not illegal to ask people to send you money and then keep the money they send you
There is a history of companies and organisations threatening legal action against security researchers when they report vulnerabilities in their systems or products.
Sometimes even when the testing has been completely offline - I know people who have downloaded some software, carried out testing against a local copy of it, and then faced legal threats when they tried to report serious security vulnerabilities to the vendor.
It's one of the reasons that some researchers don't bother trying to talk to the vendors and just go straight to full disclosure, or if they do report to vendors they do so anonymously. But if you have to pay, that's creating a link back to yourself which makes the latter much harder.
Yikes. Thanks for the good faith reply. Does EFF help to defend some of these cases?
When you report a vulnerability in a product that means you hacked the product. Hacking is illegal. If it's something that runs on your own computer you might get away but if it runs on a server then it's 100% a felony.
Sure, it sounds dumb when you say it like that.
But do you know how many people are doing things that are even dumber right this very minute? I don't know either, but I'm sure it's larger than either of us would like to admit.
why would anyone accept bounty money to have a chance of being arrested?
Yeah, I help review security reports for a small FOSS organization, and someone reported a "critical" vulnerability about a publicly-accessible SVN server. Like yes, that is indeed the purpose of hosting open source software. But at least that report was obviously bogus; much worse are the ones that look legitimate at first, so you have to read through dozens of AI-generated paragraphs to make sure that there's nothing valid hidden in there.
I use AI to read such emails!
Vulnerability reports are a voluntary service to help a vendor or software project. It’s often an annoyance for the security researcher. I understand people are getting slammed and it sucks, but the main result of rejecting them is going to be an increase in full disclosure. As a note: if you have a bug (that isn’t devastating but you’d like to talk about) having an LLM write up the disclosure is a great way to check the “we disclosed responsibly and they didn’t care” box.
> I understand people are getting slammed and it sucks, but the main result of rejecting them is going to be an increase in full disclosure.
Right, what I'm saying is that letting those bugs go to full disclosure (aka being filed as public issues, like every other bug) would have been a significant damage to user safety a year ago, and it's not anymore.
I think that’s an assumption. Just because an LLM might be able to find some bugs does not mean every attacker has a packaged attack, or the right prompting. The easiest way to find a vuln is just to Google for it.
I believe the term is Beg Bounties and they are constant and annoying.
> As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam.
I don't think that is unique to the LLM era. The company I work for has been getting some form of spam vulnerability reports years before LLMs were a thing. Often similar to what you mention about 'bad CSS'.
Maybe the volume has increased a bit, but we've added in a filtering solution and I'm more distant from the reports now, so hard to be sure.
We also get unsolicited vulnerability reports from companies trying to poach our annual pentest contract, which is... a tad grey ethically-speaking
I hated these low-effort reports, so I created a simple automation that checks my security inbox, mentions me in #security on Slack for things that look legitimate so I see them quickly, and marks things that seem entirely automated as spam.
I still check the spam folder for legitimate emails, but so far there haven't been any false positives.
but why would you answer with a C&D if you are overwhelmed? provided, it's not always the same person?
Have you considered having an agent, or just a model, classify/triage them for you? Modern problems require modern solutions.
Its been like that for half a decade across all software. People act like finding a linux kernel bug is a big deal, completely ignoring the fact that in order to exploit that bug, the attacker has to be able to run code on your computer in the first place, which is extremely hard to do these days remotely.
Also people ironically just DGAF that much. The last actual bad exploit was log4shell in java, which given how it was introduced (i.e someone purposefully at Apache made it so a log statement can execute code, and nobody questioned it before pushing it to prod), should have been the signal for everyone to completely remove all Apache libraries from their services, but yet all the software is still being used.
These bugs are indeed important, you need them once you‘ve found a bug in an application.
If someone manages to get remote code execution at user space on a machine, the amount of damage that they can do with just that versus having a kernel level exploit is about the same.
Ah yes, just move away from all apache libraries, should only take a day or two.
No I agree, its a pain, but its necessary if you care about security and don't want to audit every single release for potential vulnerabilities. People don't do this, so they really don't care that much.
Security Eng here. The whole thing is an absolute mess. I’ve been (and still am) on both sides of the fence.
I currently have two reports (one RCE on a famous OSS ML platform, one cluster take over on a k8s related projects), both are more than 2 months old without as much as an “F you, get lost”. Just got ignored and ghosted, which hurts a lot, because I spent a lot of time finding, and verifying these (all reports with poc and patch). BUT I understand why it’s happening, because I’m also on the receiving end.
security@ and VDPs have always received BS reports and beg-bounties, but boy oh boy, these days we have two people spending 3-4 days a week sifting through this constant flood of garbage compared to 2-3 tears ago where 1 person could triage the inbox and VDP in a day’s work max, which would’ve been considered very busy. Unfortunately we can’t just shutdown the programs or the mailbox because 1. We do occasionally get important and great stuff that actually matters, and 2. We’re a critical infra company and can’t ignore anything really.
The signal to noise ratio is almost zero, but the “what if” is keeping us swimming through this unending river of garbage and burning us out.
Overall, chaotic mess on both sides.
Ending on a doom-and-gloom note: there will be a reckoning.
(Don’t take the note too seriously though, I’m a SecEng, so I have a built-in doom multiplier lol)
I reported a fun security bug to Google recently (not high in CVSS terms but will probably make the HN front page when it goes public), and the report was auto-closed in minutes as "not reproducible". If they tried to use an LLM to reproduce it I'm not surprised, since it requires a soldering iron.
Having seen the other side of a security inbox I totally get it, and fortunately I was able to get it re-opened via backchannels. I think the future of bug reporting will run almost exclusively on reputation and connections.
> Ending on a doom-and-gloom note: there will be a reckoning.
Can you elaborate on what you mean by this?
I imagine:
AI vulnerability analysis is going to find something, it will be reported by a researcher and ignored as chaff, and then separately, later, someone will build it into an exploit and compromise a piece of critical national infrastructure
Have you considered using LLMs to perform some automated more-or-less reliable classification of incoming reports by severity, affected product, etc., then have agents try to replicate the reported findings?
Ideally the reports would also be coming in in the same structured format.
Security through obscurity was never a great strategy.. and now it’s not a strategy at all..
Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others.
This is an engineering problem. It won’t be solved by models that “only do dumb shit 1/10th as often, only 0.01% of the time now not 0.1%!” It won’t be solved by adding more models to do even more double-checking before and after the work. It won’t be solved by hoping humans catch it in review. It isn’t solvable by adding outer loops of any sort - though we may get close. To truly solve this will take serious CS research.
Almost never do software companies even attempt to design secure systems. I'm not sure this requires new fundamental research so much as slightly giving a shit.
There is a reason Mythos only found one bug in curl and it wasn't very bad.
Regulation and an ethics/licensing board à la Engineers would probably be a good start. If management knows they can’t tell you to do a bad or sloppy job because no one in your industry worth a damn will… everyone wins.
I just see unintended (but easily imaginable) consequences that don't fix anything.
Especially since the world isn't Dilbert where your boss goes "oh, authz? lol nah, just yolo it" and you go "dangit, alright boss". Instead, security requires eternal vigilance and zero missteps along the thousands a project takes in its lifetime.
I think there's a reason HNers who pitch this idea never give any concrete examples of entailments of their proposal: it doesn't even sound good superficially. e.g. How this actually changes security issues. In fact it just sounds even more convenient to blame engineers.
Yes. People don't seem to understand is that if we build our tools and libraries with models that do random dumb stuff 0.01% of the time, those bugs and leaky abstractions bubble up and grow exponentially into errors and undependable behavior.
People have been doing great research with formal methods, dependent types etc. Disciplining ourselves to truly write and understand code, using the best in math and PL theory (FP and type theory etc) is the only way forward in my opinion. We have to make correctness a value and goal, or else we will keep spinning off into psychosis and the corruption of truth.
At the end of the day security problems are just bugs like any other. If the resources to find them outweight their value then there is no sense in finding them (other than to give security researchers' jobs)
I agree they’re bugs, but the impact being far greater should justify special treatment - and it does. You’ve got bug bounties, special access and even special devices for security researchers, if you count Apple’s program where they’ll let research be done on a special iphone.
The resources to find security bugs should be weighed against the impact of them being found & exploited by bad actors, not weighed against spending those resources on other things. (And small/simple products won’t suffer as much if a problem is found, so those cases justify smaller security investments)
Verifying correctness of an implementation is P NP, not serious CS research.
Rust's safety guarantees are also undecidable on an arbitrary program, but Rust still proves them for safe rust code. Why so? Because safe rust code is not an arbitrary program, it is specifically forbidden from doing some things, which becomes a pain in the ass sometimes for a coder, but the result is a subset of programs you can prove safety guarantees for.
The issue is how to design a language that restricts programmers in a way you can prove everything you care for on their programs, and still don't make life of programmers unbearably difficult, because the language allows them nothing.
Most verification is undecidable, lots of it is pspace complete. That doesn’t mean very much in practice since those are worst case bounds. People regularly solve problems that are undecidable for all practical instances that they care about.
Verifying behaviour of an arbitrary program is uncomputable. However that doesnt mean you can't have proofs of behaviour of specific programs you create.
Personally i have some doubts, a lot of research has gone into the idea without much to show for it, but its a very reasonable research area.
There's lots of things to show for the research!
Part of what the research shows is that correctness-by-proof has a cost in developer effort.
If there really is a vulnerability-apocalypse due to AI, and it's not just a different flavour of AI hype, the cost of having insecure software will rise to the point that the cost of dealing with insecure or incorrect code at time of creation becomes less than the cost of ignoring it until it blows up.
I doubt it'll rise so much that we'll want to face the cost of behaviour proofs for much code at all, but it's quite possible it'll rise enough that we want to do things like prove that indices are in bounds, at compile time, so vector accesses can skip checks without compromising safety.
I fear it'll just move the problem one layer up. Sure, you've now proven that the code matches the specification - but how do you ensure the specification is watertight?
The specification doesn't have to be.
But yeah, writing specs is usually harder than reviewing the code 4 times :)
It kinda does.
See WPA2 KRACK, you could've had a formally verified WPA2 implementation and it still would've been exploitable because the flaw was the specification itself.
Moving the problem one layer up is the desired outcome because it makes exploitation much more difficult. Your attack surface becomes much smaller.
It might just mean the opposite. Unergonomic and/or slow memory safe languages might not be needed anymore at some point, because LLM can check for reasonable programming patterns and can do some amount of vulnerability checking upfront. But seriously the first point -- as long as you keep to a known set of reasonable patterns (much larger set than the sets accepted by restrictive and cumbersome type systems), memory unsafe languages are actually pretty safe, and the code remains easy to understand, easy to maintain, and performant.
Already today, in my experience, widespread models like Claude 4.6 or 4.8 can quite reliably find some concurrency bugs that are easily missed by humans.
> can reliably find some
Some.
In software & in security, 99% is a failing grade.
So is 99.99%, so is 99.999%, and any other amount less than 100%. It’s not enough to point 5 LLMs at it and it’s not enough to point 500 LLMs at it.
The field needs to seek deterministic & comprehensive solutions to whole problem classes.
Okay, so the whole world is failing and success doesn't exist.
Maybe now you need a better criteria than your previous failure / success_100%_infiniteCost to observe and understand the the world
This logic enables and encourages corruption in all spheres of life. "Things are messed up as they are, so why try so hard to do the right thing?" "Let's be realistic."
The logic allows that to continue because "good" and "better" isn't good enough, only "perfect" is allowed, so instead "nothing" is done
It's not as black and white as you claim. 99% is obviously much better than 0%, and 99.999% would be an exceptional, unheard-of improvement. In most software projects, this would find ALL bugs.
And most "memory safe" languages are not all that memory safe. Or they need escape hooks.
EDIT: To be clear it's not AT ALL black and white. You have drunken too much of the type safety kool-aid.
I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special.
Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to minimize creating bugs in the first place. That could be new or better languages, less 3rd party dependencies, more vetted code, better linters, better fuzzers, whatever. The point the new reality of bugs being easy to find will, actually must, lead to less bugs eventually because the world can't function with easy to find bugs.
"Temporary" can be an awfully long time. There is ample evidence that discovery rate of bugs (many of which can be bucketed into vulnerabilities) in any non-trivial piece of software is more or less stable.[0] In a recent podcast episode the ex-CISO of Adobe commented that every now and then they'd take a sustained squeeze to find all occurrences of a given type of bug (ie. source of vulnerability) in a codebase. They'd find a good amount of them and fix them.
Then a year or two later they'd repeat the operation and they'd find about the same amount of same types of bugs. In many occasions in code that had been in place in the previous round and had remained essentially untouched.
Paraphrasing what the Gruqg has quipped - a large piece of software has infinity bugs. Infinity minus N is still infinity.
0: Discovery rate with regards to the time spent looking for bugs. LLM-powered bug hunting has amped up the speed with which code bases can be investigated.
Ahhh - you are talking about Adobe. I always wondered, given the never ending stream of vulnerabilities in their products, what it was about their development process that produced such appalling code in the first place.
The hope is that LLMs can scan my code every day or something like that. If I make a mistake and get it past code review, the LLM will still find it and it gets fixed right away. (better yet, make LLM an automatic reviewer on everything).
Many of the bugs we are finding in projects like curl are 20 years old - once they are fixed they are fixed and so hopefully we get all those 1-20 year old problems fixed and future scans only find new problems which is itself a big improvement in the rate. I agree that we will never reach a point where there are no bugs introduced, but we should strive to fix them faster.
I feel this sentiment is wishful thinking,but I want to start by saying I hope it turns out to be correct.
I find that often bugs will be created when using an LLM, like others have said. Saying that this can then be fixed by identifying all the bugs created by an LLM with an LLM doesnt guarantee another bug is not introduced when the LLM is addressing the initial problem.
Also, what if the LLM has a blind spot. They certainly also could be incapable of finding or fixing a bug. They dont pass any benchmark at 100% right now. Also also, guaranteeing there are no bugs in your code is like saying you have 100% test coverage, all of the tests pass, and they are written perfectly. Saying that you can simply identify and fix the bugs also assumes there is enough time and energy to find all of the bugs that exist within a project and then to address them. Even LLMs use time and energy. In a sufficiently complex system that is certainly wishful thinking.
Considering the size and complexity of a lot of modern software (like web browsers, 3d modelling software, game engines, etc.) software is just too complex to not have bugs even when created and managed by LLMs.
There will continue to be bugs in code and we will simply have to live with the fact that LLMs make it easier to exploit computer systems. I mean consider a hardware bug like Spectre [0]. If bugs like this become easier to find does that mean our existing hardware will just become obsolete more quickly? that type of problem can be addressed, but at quite a high cost.
Not sure what all of this means for the future.
0. https://en.wikipedia.org/wiki/Spectre_%28security_vulnerabil...
If LLMs can trivially find bugs, then they can trivially find bugs. If they can't find any bugs that doesn't mean there are no bugs but it suggests that others can't easily find them either. So the LLMs find all the bugs problem is fixed by asking the LLMs to find them before you ship.
Read what wrote, I didn't say your program will be bug free. I said, if the LLM can trivally find the bug it will. If it can't then we're at worst, back to the state of before LLMs could find bugs, but likely much better since we fixed so many of them
So, the fact that LLMs can trivially find bugs is enough to get the bugs fixed.
You, and several others, seemed to think I was saying LLMs would fix all the bugs. I never said that. I said they'd help. Finding them is help. Writing a possible fix is often help. Writing a possible fix and seeing if they can detect a bug after the fix is applied is also help. Automating the entire things and letting LLMs fix them without review is likely not help.
The bugs I can trivially find, the bugs Claude can trivially find and the bugs Codex can trivially find are not necessarily the same. The most obvious bugs would be obvious to any of us three, but beyond that we would't agree on a definition of 'trivial bug'.
This gets worse if you factor in different harnesses built for the task, and future model updates
Sure, things will get calmer than they are now, but shipping without bugs someone else considers trivial will still require more effort than most are willing to invest
But by using a LLM everyone is automatically devoting that much effort, and so while we don't fix all bugs we fix a lot that wouldn't have been.
Ok, I get a better idea of what youre saying from this reply than your original comment. It wasn't helpful to me that you suggested I reread your original comment.
I agree that LLMs make finding bugs take far less time and energy. I also agree that this should mean in the long run there are less trivial to find bugs IF everyone adopts the usage of LLMs while writing and reviewing code.
It does also seem possible that LLMs are better at finding bugs than fixing them.
This is where you're wrong, I ran an experiment and told it to find bugs in a ~200 LoC project. The models are tuned in a way to where they're expected to generate issue reports so a codebase that had zero bugs, zero vulnerabilities and zero changes needed it found 3 low severity issues (cosmetic) 1 medium severity issue and 1 critical severity issue. The critical severity issue was accepting unvalidated user input, for... an echo command.
Did you make any attempt at tuning the prompt to reduce false positives? Or did you just say "find bugs"? Because if you tell it to do that, it will.
The point I was trying to make is that there will always be people reporting "critical" bugs.
That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.
How can it be easier to find them than to not create them? Whatever you do to find them, you could do before you release.
Because the behaviour of software changes over the time of development and that's how many bugs happen in the first place.
Especially if you use AI, let's say you have it implement a feature and then change your mind. In my experience AI makes as many if not more bugs than a human.
You can accidentally create a bug that you yourself cannot find.
What's the difference between finding bugs and not making? Just run the bug finding in during CICD.
It’s not necessarily symmetrical, and in fact would be very surprising if it was. It’s a probabilistic algorithm on both sides, so the energy use to find any working program vs all bugs in a working program are fundamentally different search spaces. Not to mention the false positive rate and the human verification effort. Then even the idea of incremental security checks is potentially flawed since many security issues are non-local (ie not localized to a single module).
I think I’m on this side. I find it exceedingly unlikely that we just start producing “perfect” software all the time for everything, and at the same time start generating an order of magnitude MORE software.
It does not suppose that LLMs can write secure software
> That supposes that LLMs can write secure software.
I think we're at the point that the best LLMs can indeed write software that's far more secure than your average programmer. Partly because the average is so terrible.
There's an assumption in here that every developer is spending a load of money on the latest and most capable LLMs to scan for bugs in their code before every release.
But the last couple of decades have shown us that huge numbers of developers aren't even following basic and free secure development practices, let alone pouring money into expensive scanning tools.
It may be only a matter of time before all devs remember to append ", and make no mistakes" to the end of their LLM prompts, but I don't think we as an industry will ever reach a point at which every release of every package/library/application is scanned with the most capable model available.
I mean, we've had tooling like fuzzers available for a very long time, and most devs haven't run one against their software ever, let alone before each release.
It's the human factor I think will keep this a problem essentially-forever.
LLMs are finding bugs where there aren't any and wasting human time trying to disprove the slop.
If all LLM reports were accurate, they'd be of any value. However, that's not what is happening. If you have even mentioned something about a bug bounty anywhere, waves of slop peddlers will flood you with fake reports marking every minor bug as a critical problem, hoping to catch a handful of dollars in the process.
These models do find some problems and may even provide decent suggestions to fix them (though they really want to add code above anything else, quickly leading to spaghetti if you accept it all). That's not the issue at the moment, and as long as people try to incentivize people to report bugs, the issue will remain.
I do expect this to be temporary, though. Not because LLMs will fix all the bugs, but because the flood of slop will shut down most public bug bounties.
thats is definitely NOT what the article says.
Are you making a counterpoint that the reports are so good and must all be addresses, but the problem is "llm finding all the bugs" so fast us poor slow humana cannot keep up?
because if so, i suggest you write a new article.
> Once most of the bugs are fixed
my brother in christ, I hope you're actually trying to be funny here.
99 little bugs in the code
99 little bugs
take one down, patch it around
105 little bugs in the code...
Lol you think LLMs are generating bug free code?
I never said that. I said they are good at helping fix them. Go read the bug reports on firefox, or Safari, or Chrome. Most of them have a fix. It might be wrong but it usually points in the right direction, which is a 1000x more than nearly all human bug reports. So, the LLM helps. which is all I stated.
At risk of quoting too much of the article, it opens with this:
> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all.
> Except…
> For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t apply to vulnerability reports. No, vulnerability reports are special. Security researchers are doing us a favor by reporting things confidentially instead of doing full disclosure, so we owe them something, which is not true of regular issues opened on the issue tracker.
[...]
> It’s 2026 and none of the premises are true anymore.
I respectfully disagree.
The premise is absolutely still true: if someone discovers a critical, exploitable vulnerability in your software, the impact and tradeoffs are exactly the same as they were before LLMs started finding bugs. There are just more of them now, so they're easier to come by.
But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
All of that said, I don't think finding vulnerabilities has really been the difficult security problem for most companies (or open source projects). The difficult problem is dedicating resources to fixing those vulnerabilities instead of building software, products, and/or infrastructure that people want. That problem is absolutely still here today, but I'm optimistic that agentic security developers will be able to take the burden off of development teams in the near future.
For tokens, of course.
The problem is there used to be a fairly high correlation between ‘security report’ and ‘real vulnerability’. Not perfect but good enough. Now the two are almost entirely disconnected.
> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
I think your logic is partly correct but the fact that the same LLMs are allowing an exponential increase in insecure code generated is a counterbalancing point. I do not think this phenomena will slow down.
Nah, those same LLMs, if prompted correctly, will be able to do an audit pass and a fix pass on that LLM-generated code. It’s a tooling issue that will get fixed in time.
> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report.
That is not my experience at all. People will continue to high-volume spam intended behaviour as if it is a bug.
There will be fewer reports that matter as you fix things - but the volume of reports will either stay steady or go up. Making it harder to even notice the ones that matter.
The problem always existed, but nobody amassed a sufficiently large army of trolls to exploit it until now. So it wasn't a priority to solve it before, but now it is. We're going to have to learn to differentiate reports that matter from those that don't. Classifying reports might actually be something you could productively use an LLM for..
When we solved this problem for email... We just dumped everything similar to untrusted into the rubbish bin. Important things can vanish too, and that's an acceptable price.
Yeah unfortunately there will be some high quality reports that get erroneously discarded, but so long as the signal/noise ratio improves sufficiently that's an acceptable price.
It's not (just) more of them, it's the same ones reported by multiple people.
I think the point is those issues are now easily discoverable and are nearly public because of it.
LLMs find more vulnerabilities than people because people time is heaps more expensive than LLM time, that’s it.
We’ve always been able to find heaps, we’ve just never had the right structures to put in the effort and renumerate people for looking (even if they don’t find anything).
Looking for bugs is boring. LLMs don't get bored the same way humans do. Thus even if you had infinite budget you should expect a LLM to be better.
There are some problems with incentives in the vuln report space. People report trivial vulns and expect the same treatment as people reporting critical vulns. But this isn't new with AI. Look at all the ReDos vulns in npm ecosystem. Its questionable if its a vuln in general but half of them aren't even triggerable.
They weren't special even before LLMs. Drive-by script-kiddies would run some basic scripts against your platform and send generally-not-actually-a-vulnerability reports, claiming that these were big problems, and requesting to be paid bug bounties.
And then they submit them to a CNA and get a CVE assigned, and then _everyone_ needs to deal with the not-actually-a-vulnerability report, especially when the not-actually-triggerable-DOS gets assigned a "Critical" CVSS score from EUVD or NVD.
I agree with this. One of the consequences of the "vulnpocalpyse" is that it's become even harder to sift through the noise: I triage well over a dozen reports a week, many of which are "real" in the sense that they reflect a genuine defect but otherwise have an unclear impact on a typical user. This has always been true of the median vulnerability report, but the volume means that I now lean much more heavily away from coordinated disclosure.
One flipside to this is that, because many of these bugs are "shallow" to LLMs, it's actually easier than ever to moderate the worst participants in your vulnerability program -- if someone sends you slop, you can just ban them and wait for the next, better orchestrated LLM to send you a better report for the same vulnerability.
this is hilarious and i might try it.
I read every piece like this one as: Money is moving in the vulnerability space now, when as before the LLM hype incentivized that, your best bet was that someone skilled enough would accept living with the financial insecurity of being a gig worker to hopefully stumble upon your projects bug bounty program. Is the bet here is that the hype lasts, and that people willingly will keeping on paying Dario to be able to contribute?
> But give it 1-3 months and the open models will catch up.
I wish that this would stopped being thrown around, what is this timeline based on? How good is your open model from between March and May?
Also, having read "Gödel, Escher, Bach" I know that the hare never catches up with the turtle.
> LLMs are as good as almost any security researcher, and anyone can run them.
I wonder what the metrics are. Also, not "anyone", just the affordable.
Vulnerability reports were never special.
The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now.
Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstration, the communication that was special.
When you realize that this is being written from the perspective of someone who does vulnerability reporting in a professional capacity, you’ll connect the dots. We took care to be kind and succinct because for many of us, we learned our skills from being on the development side of things first.
Vulnerability reports aren’t special anymore. The only ones that felt special were the ones with human touch, the ones doing their job as an adversarial thinker, and taking the care to understand that net positive outcomes require coordination even if both parties don’t see eye to eye.
Nothing has changed. It never was. You’re just inundated with AI slop; which as a practitioner who uses AI regularly I can say with absolute confidence. The end result is the same, the volume is increased, but the special thing was never the report itself.
Finding a vulnerability was always the easy but high toil part. It was the care to communicate succinctly and be invested in the outcome that was special.
Godspeed.
This x 1000
I’ve been screaming this from the rooftops. Impact is what was always important. No one is going to take down prod to do an emergency patch on an RCE that COULD NEVER ACTUALLY BE EXPLOITED.
I feel like we’re witnessing the result of multiple roles suddenly becoming security aware but not having the background or understanding to make any sense of it.
In an ideal universe yes. But we live in a world where vulnerability scanners reign supreme.
Yep, I've updated dependencies with an RCE that can't be exploited in my codebase just to keep my security team happy. Not worth the multiple arguments about it not actually being an issue.
You can never guarantee that the codepath of a dependency that is vulnerable can not be reached or used as a gadget in an exploit chain. Patching dependencies, even when no direct vulnerability arises is an essential part of defense in depth and sevurity hygene.
I don't disagree, though I should have been more specific - I updated dependencies at extremely short notice because that's what security wanted. I would have updated the dependencies sooner or later anyway because I know that periodically updating dependencies is generally a good thing to do.
You can also never guarantee the patched software doesn't include a worse vulnerability, I would submit that patching software without proper time to validate changes is also a security issue.
If you aren't careful, that is how you get this security theater.
This screams LLM to me and I couldn't bear to read past the second paragraph.
I read a casual teams message from one consultant type of guy who said "the low level linux hackers who don't care about best practices, code quality and security get what they deserve". I don't know how to feel about this other than disgust. The entire space has been overrun by LLMs making stuff up or finding things that aren't critical while some roles capitalize on the fear. There is just so much wrong with saying such things in such a way, I couldn't describe.
This entire demeanor of the way people and ai people talk about software and technical people is getting absurd and simply unprofessional. It is as if we stopped being adults.
Cannot agree more. As part of our SOC2 we have to log and respond to all inbound vulnerability reports. Before it was easy to tell if a report was just a bounty hunter looking for a low hanging fruit. Now well crafted emails with seemingly legitimate disclosures take a lot of time to validate and triage.
Our solution was to build a tool that uses LLMs to assess the report before it gets to us. Honestly I wish we didn’t have to do this but it works and has really allowed us to spend our time on the actual good reports. (Feel free to check it out at fortworx.com if you want)
These two bits stand out to me:
> The security researchers are not special, the insight and confidentiality are
vs
> The bottleneck now is not finding potential issues but assessing which ones are real. Unless there’s already a trust relationship, external researchers can’t meaningfully contribute
My take-away from this is that the researchers were special all along and you should probably be building a trust relationship with them.
Despite what I want to believe about tech being a meritocracy, the reality is that trust plays an extremely important role and without it we risk a collapse of our open source software ecosystem.
One of my biggest criticisms of AI is the trust vacuum within which it operates
Can anybody say what is going to happen? It's not a rhetorical question, and the implied or entailed context might involve a Nash equilibrium of some sort.
Right now the rate of signal is high, and the ratio of noise is proportionally high. But it seems like everyone expects the signal to eventually plateau or sharply decline. Almost as if there is a finite supply of "low hanging fruit" for shallow scanning machines to easily discover, and then there will be some kind of new world that follows where only truly difficult problems emerge.
But eventually the question then becomes why even bother with Rust or any other silly borrow checking ideas if we can use more enjoyable programming languages with LLM side-kicks to catch security vulnerabilities on the front side of the development workflow?
IT seems to me if we exhaust all the extant security vulnerabilities to a calculus that asymptotically goes to infinitesimal zeroness, then... the only trick remaining is to scan code before it becomes vulnerable.
I'm wondering whether this is a permanent change. After all the easy-to-find bugs have fixed and you can't find them just by asking an AI, perhaps security issues will deserve special treatment again.
I wonder if LLM's 'jagged intelligence' will come to bite here again. AI might be better at finding certain kind of bugs, to the degree outperfoming the best humans, but that doesn't mean a skilled human can't find an issue which is hard for LLMs incredibly easily.
Were they ever "special"?
The inbound stuff we get through the vulnerability e-mail is pretty much exclusively spam. Then they start e-mailing random people of the company they can find to get through, and in the end it's still spam.
Linus Torvalds once went on record saying security vulnerabilities are no more important than regular bugs.
This of course made vulnerability researchers seethe worse than aggrieved Redditors.
It turns out he was right all along.
The author also gets it wrong by assuming that regular bug reporters are not "providing a service". They are.
When I wrote up a bug report, I made sure it's thorough with detailed steps to reproduce. It takes a lot of time and I've done it professionally for projects you've absolutely heard of.
Having said that, getting them ignored repeatedly and — even worse — having my detailed PRs rejected, sometimes within minutes, as if I'm some ignorant luser is why I don't do it anymore. My time is more valuable than your hubris.
A lot of open source developers have their heads so far up their own asses they forgot that it takes a community for projects to be successful.
> Linus Torvalds once went on record saying security vulnerabilities are no more important than regular bugs
Linus also regularly calls the contributions to Linux kernel "garbage". These contributors, btw, are the reason that autist doesn't have to work for a living, where he'd find out what the real world is like
Stella Ops has call paths, vulnerabilities classifications, VEX to mitigate most of these: https://stella-ops.org/features/
In the end, sorting prs and vulnerabilities has been the same for open source maintainers. How about adding a credibility score to every github account ? Couldn’t that cut sorting times ?
This whole blog post makes me sad. I've been active on both sides of the vulnerability disclosure process for well over a decade and have reported a whole bunch [1] of security bugs to the Go security team. I was there back when Filippo was running the show and have continued since Roland took over. My experience with the people there has always been great.
> Ultimately, it all stems from our responsibility to our users. The security researchers are not special, the insight and confidentiality are, and we need them to keep our users safe. Ignoring a security report communicates you don’t care about users’ security, and it’s rightly a reason for shame.
100%. This was always true and I still think it is. LLMs don't change anything. At most they shift the balance and force a temporary compromise.
> LLMs are as good as almost any security researcher
This statement is extremely dependent on the definition of a security researcher. It might hold if you consider anyone with a HackerOne account, but if you restrict the definition to people who actually put in some effort, it's just not true. LLMs can find some real vulnerabilities, yes, but they also spew unprecedented volumes of garbage that an expert can immediately recognize as such.
> The insight is not scarce and precious anymore. The bottleneck now is not finding potential issues but assessing which ones are real.
Assessing which ones are real should be part of the insight. Real researchers will not submit 150 pages of spam, and three real bugs hidden in 150 pages of spam are not insight. In most cases a researcher will spend significant effort on triage before submitting anything, and an LLM still cannot do that reliably.
> Confidentiality, embargoes, and coordination also don’t matter nearly as much as they used to.
I'd argue these now matter more: the one thing LLMs do seem to do fairly well is figure out specific things based on sufficient information and a scope that's limited enough. So a plain commit containing a security fix is now much easier and cheaper to turn into an exploit than it was before.
> The years of vulnerability reports being special might be over, as weird and uncomfortable as that feels.
I'd hope not. Bug bounties might be over unless someone can figure out the spam problem, but disclosure programs that don't offer monetary incentives are probably just going through a tough period that will eventually calm down as the operators of LLMs realize the costs and do the math.
Unreliable reports have always been an issue and will remain one, LLMs or no. When it gets worse, like in the current influx of LLM-generated reports, the focus should be on identifying reliable researchers, building relationships, and providing guidance on how they can make the reports easier to triage.
Researchers are not special, but the insight they can provide totally is. LLMs might force everyone to make better use of that insight, instead of just consuming bug reports and drowning in triage.
[1] https://groups.google.com/g/golang-announce/search?q=juho
Thanks for the comment, I was actually hoping to get your take on this! I linked to it from the article.
> Still on Hacker News, Juho Forsén, one of the most prolific reporters of Go security issues, wrote a long interesting comment that makes the argument that instead we should lean harder into trust relationships with individual researchers. It'd certainly be worth it with Juho, in retrospect, but it's unclear if it would pay off often enough, in the same way that training new contributors who might leave the project in a month or two is not always worth it.
I've run a disclosure program for ~7 years, which is an open paid program. However, over that time we've developed relationships with the most active and successful contributors, to the point that we'll now give them early access to new features to try out (all still paid for on the basis of rewards for problems found). This is proving especially valuable now in triaging the new deluge of noise from impactful issues.
I wrote about this this morning [1]:
> We're keeping our vulnerability disclosure program open - because even though they are rare, the genuine critical reports we receive, in amongst the noise, are still highly valuable. I don't think we're at the stage yet where finding those issues is a purely mechanical process; persistent, imaginative researchers still make a contribution to the process by finding things that LLMs by themselves, so far, haven't.
[1] https://www.linkedin.com/feed/update/urn:li:activity:7475447...
I found a DoS vulnerability in Coinbase several months ago on Hacker One. It took me literally 30 minutes to find. First time I did this in my life. I could craft a message cheaply which, when sent as the HTTP payload to a specific endpoint, would cause the server to hang for a full 30 or so seconds before getting a response. I could have easily scaled up that attack, cheaply...
I filed a report, they marked it as 'informative' and thanked me, recommended I keep looking for more vulnerabilities, but no payment at all; they said I had to be able to demonstrate major disruption of service... Which I presume is illegal. I literally showed them all the ingredients of the attack, the exact curl commands, payloads, the exact response delay could be easily be verified; you could see the server response slowing down proportional to the degree of nesting in the payload. I could execute it without authentication too; so it was essentially certain that the attack could be scaled but they made it impossible to get a reward.
The hardest part was writing the report which took several hours.
So yeah, 30 minutes of looking for a vulnerability, no prior experience in security research, first project I looked into on Hacker One, ever... A company in crypto sector which is a major target of hackers and takes security relatively seriously.
Imagine how insecure most software is! Imagine how bad most vibe-coded software is especially! Companies might as well run their servers directly inside Kim Jong Un's data center in North Korea.
North Korean hackers probably have a dashboard which shows more detailed and accurate platform analytics than what the founders of the company can see.
I think the main impact of this is that a successful career in vulnerability research is going to require a high level of proficiency in exploit development, as that's where the demonstrable real-world impact lies.
Tech careers no longer exist. Tech jobs will still exist for a few years, but careers they will no longer be.
Honestly, even that is something LLMs are becoming scarily good at. Not all of it, but they have surprised me, including in terms of exploit development, more than I am comfortable with.
These kinds of posts act like there wasn't already a rich suite of automated security scanning tools available. The fact you can put them in a LLM's hand or point it at a codebase to do the exact same thing isn't really a big leap, the spam is. It's a bunch of amateurs thinking they can do work outside of the realm of their expertise now, and this trend isn't unique to security, you see it spreading across the various realms of expertise out there in the world. The world's slowly filling with slop, and CVE spam is a byproduct of that.
I'm curious about people's experiences with Kalshi support in this context.
Tangent point, I think more broadly this is a big piece of AI-cynicism in general- “x isn’t special anymore”.
It’s tough staying motivated on a craft when an AI is nearly as good as you. Chess players manage to do it at least.
> Chess players manage to do it at least.
The 5 on earth still getting paid to play chess?
There's only one Magnus Carlsen, who earned > $1 million in 2025 for playing chess, but the long tail, there were 26 people who made more than $100k, https://thechessworld.com/articles/general-information/the-1...
but like, if you mean literally "someone gave them money and they played a game of chess", the number becomes much bigger. Chess coaches, streamers, club instructors, exhibition players, league players, camp counselors, and titled players receiving appearance fees, etc. All told, you're looking at ten's of thousands across the world.
And if you mean “people who can live off of tournament winnings” it’s not more than 26.
It’s like most of art, writing, and sports. The only way to make money is by becoming a teacher.
"LLMs are as good as almost any security researcher"
Oh really? If LLMs were as good as almost any security researcher then you wouldn't be getting flooded by bullshit reports from them. You'd be receiving legitimate reports instead.
Really. Back in the days, the majority of “researchers” sent scan results. The term “beg bounty” didn’t come about for nothing.
"LLMs are as good as almost any security researcher"
No they are not. Everything else can be safely ignored. The author is suffering from AI psychosis and needs to get some help.
Everyone here is apparently, that's why you getting downvoted.
Yet again it's <person who is not an X> saying that AI can replace the job of an X. Take a hint people, if AI can't replace your job it can't replace anyone elses'.
>> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation.
I don't think the gift analogy works well. In most cultures, turning down or even ignoring a gift is considered anywhere from impolite to hugely offensive. But that's the opposite of open source: there's nothing wrong with requesting changes to a PR or even closing it.
Plenty of people offended by closing a PR or issue unresolved.
>I honestly have no idea how the profession will look after that, so this whole post is more of a current observation than a long-term prediction.
My sweet summer child, the "profession" will become an agent managers talk directly to, like many other professions.
> If a security vulnerability is reported by someone who is also violating the CoC, what do you do? Do you ignore it? Fix it silently?
Is this even a question? You triage and fix the vulnerability just like any other one. Are truths spoken by folks one dislikes — even for perfectly valid reasons — any less true?
The only way I can imagine this somehow applying is if someone has a habit of reporting vulnerabilities which do not exist, or of exaggerating their severity. Is crying wolf a CoC violation? If so, then I can imagine that particular sort of bad behaviour justifying some consideration before acting on a report.
Will xorg backport patches from Xlibre?
No, because xorg is a dead project that doesn't take any patches from anywhere and xlibre has shit code quality and is probably vibecoded now
really? From what I have seen so far most of the contributions to xlibre have been cleaning up stuff thats been obviously wrong in xorg, and using tools - including AI and more bog standard tooling to find issues, add safety checks, and a whole lot more. They're merging in a namespace extension which solves many of the purported 'security' issues in X11 today.
How badly are they violating the code of conduct? It wouldn't be the first time a security researcher got thrown into prison or jail, in this line of work.
soo true
I guess we now need AI tools to filter security report spam. I'll go out on a limb and say such products already exist.
> LLMs are as good as almost any security researcher, and anyone4 can run them.
What is this, rage bait? It's bullshit, and insulting to actual security researchers.
That might be true for low-effort vulnerabilities and fake security researchers, but the real security researchers are far from being replaced by LLMs.
I agree. Accoding to a security engineer I know, the impact of mythos is enormous.