PSA: npm/bun/pnpm/uv now all support setting a minimum release age for packages.
I also have `ignore-scripts=true` in my ~/.npmrc. Based on the analysis, that alone would have mitigated the vulnerability. bun and pnpm do not execute lifecycle scripts by default.
Here's how to set global configs to set min release age to 7 days:
(Side note, it's wild that npm, bun, and pnpm have all decided to use different time units for this configuration.)
If you're developing with LLM agents, you should also update your AGENTS.md/CLAUDE.md file with some guidance on how to handle failures stemming from this config as they will cause the agent to unproductively spin its wheels.
I actually think it is not too bad a design, because seconds are the SI base unit for time. Putting something like "x days" requires additional parsing steps and therefore complexity in the implementation. Either knowing or calculating how many seconds there are in a day can be expected of anyone touching a project or configuration at this level of detail.
Seconds are also unambiguous. Depending on your chosen definition, "X days" may or may not be influenced by leap seconds and DST changes.
I doubt anyone cares about an hour more or less in this context. But if you want multiple implementations to agree talking about seconds on a monotonic timer is a lot simpler
I came here to argue the opposite. Expressing it in seconds takes away questions about time zones and DST.
I think you're incorrect to say that second are also ambiguous. Maybe what you mean is that days are more practical, but that seems very much a personal preference.
that kind of complexity is always worth it. Every single time. It's user time that you're saving and it also makes config clearer for readers and cuts out on "too many/little zeroes on accident" errors
It's just library for handling time that 98% of the time your app will be using for something else.
Workdays! Think about it, if you set the delay in regular days/seconds the updated dependency can get pulled in on a weekend with only someone maybe on-call.
(Hope your timezones and tzdata correctly identifies Easter bank holiday as non-workdays)
In JavaScript something entirely new would be invented, to solve a problem that has long been solved and is documented in 20+ year old books on common design patterns. So we can all copy-paste `{ or: [{ days: 42, months: 2, hours: "DEFAULT", minutes: "IGNORE", seconds: null, timezone: "defer-by-ip" }, { timestamp: 17749453211*1000, unit: "ms"}]` without any clue as to what we are defining.
In Java, a 6000LoC+ ecosystem of classes, abstractions, dependency-injectables and probably a new DSL would be invented so we can all say "over 4 Malaysian workdays"
In before someone thinks it's a joke, the most commonly used logging library in Java had LDAP support in format scripts enabled by default" (which resulted, of course in CVE)
But you know that Java solution will continue working even after we no longer use the Gregorian Calendar, the collapse and annexation of Malaysia to some foreign power, and then us finally switching to a 4-day work week; so it'd be worth it.
... and since it was architectured to allow runtime injection-patching of events before they hit the enterprise-service-bus, everyone using this library must first set fourteen ENV vars in their profile, and provide a /etc/java/springtime/enterprise-workday-handling/parse-event-mismatch.jar.patch. Which should fix the bug for you.
You can find the patch files for your OSs by registering at Oracle with a J3EE8.4-PatchLibID (note, the older J3EE16-PatchLib-ids aren't compatible), attainable from your regional Oracle account-manager.
JavaScript Temporal. Not sure knowing what a "workday" is in each timezone is in it's scope but it's the much needed and improved JS, date API (granted with limited support to date)
Don't forget about regional holidays, which might follow arbitrary borders that don't match any of the official subdivisions of the country. Or may even depend on the chosen faith of the worker
That had more or less been the explanation in the books for decades, and even in George Lucas' notes from 1977:
> It's a very simple ship, very economical ship, although the modifications he made to it are rather extensive – mostly to the navigation system to get through hyperspace in the shortest possible distance (parsecs).
They made a movie to make money. I doubt anyone holding the purse strings cared one iota if that bit were corrected or not. It’s not really a retcon either because they didn’t change anything.
To me it sounds safer to have different big infra providers with different delays, otherwise you still hit everyone at the same time when something does inevitably go undetected.
And the chances of staying undetected are higher if nobody is installing until the delay time ellapses.
It's the same as not scheduling all cronjobs to midnight.
About the use of different units: next time you choose a property name in a config file, include the unit in the name. So not “timeout” but “timeoutMinutes”.
At that point, you're making all your configuration fields strings and adding another parsing step after the json/toml/yaml parser is done with it. That's not ideal either; either you write a bunch of parsing code (not terribly difficult but not something I wanna do when I can just not), or you use some time library to parse a duration string, in which case the programming language and time library you happen to use suddenly becomes part of your config file specification and you have to exactly re-implement your old time handling library's duration parser if you ever want to switch to a new one or re-implement the tool in another language.
I don't think there are great solutions here. Arguably, units should be supported by the config file format, but existing config file formats don't do that.
> adding another parsing step after the json/toml/yaml parser is done with it. That's not ideal either
I'd argue that it is ideal, in the sense that it's the sweet spot for a general config file format to limit itself to simple, widely reusable building blocks. Supporting more advanced types can get in the way of this.
Programs need their own validation and/or parsing anyway, since correctness depends on program-specific semantics and usually only a subset of the values of a more simply expressed type is valid. That same logic applies across inputs: config may come from files, CLI args, legacy formats, or databases, often in different shapes. A single normalization and validation path simplifies this.
General formats must also work across many languages with different type systems. More complex types introduce more possible representations and therefore trade-offs. Even if a file parser implements them correctly (and consistently with other such parsers), it must choose an internal form that may not match what a program needs, forcing extra, less standard transformation and adding complexity on both sides for little gain.
Because acceptable values are defined by the program, not the file, a general format cannot fully specify them and shouldn’t try. Its role is to be a medium and provide simple, human-usable (for textual formats), widely supported types, avoid forcing unnecessary choices, and get out of the way.
All in all, I think it can be more appropriate for a program to pick a parsing library for a more complex type, than to add one consistently to all parsers of a given file format.
Another parsing step is the common case. Few parameters represent untyped strings where all characters and values are valid. For numbers as well, you often have a limited admissible range that you have to validate for. In the present case, you wouldn’t allow negative numbers, and maybe wouldn’t allow fractional numbers. Checking for a valid number isn’t inherently different from checking for a regex match. A number plus unit suffix is a straightforward regex.
Is there a way to do that per repo for these tools ? We all know how user sided configuration works for users (they usually clean it whenever it goes against what they want to do instead of wondering why it blocks their changes :))
If everyone or a majority of people sets these options, then I think issues will simply be discovered later. So if other people run into them first, better for us, because then the issues have a chance of being fixed once our acceptable package/version age is reached.
min release age to 7 days about patch releases exposes you to the other side of the coin, you have an open 7 days window on zero-day exploits that might be fixed in a security release
At least with pnpm, you can specify minimumReleaseAgeExclude, temporarily until the time passes. I imagine the other package managers have similar options.
Urgent fix, patch released, invisible to dev team cause they put in a 7 day wait. Now our app is vulnerable for up to 7 days longer than needed (assuming daily deploys. If less often, pad accordingly). Not a great excuse as to why the company shipped an "updated" version of the app with a standing CVE in it. "Sorry we were blinded to the critical fix because set an arbitrary local setting to ignore updates until they are 7 days old". I wouldn't fire people over that, but we'd definitely be doing some internal training.
Their analysis was triggered by open source projects upgrading en-masse and revealing a new anomalous endpoint, so, it does require some pioneers to take the arrows. They didn't spot the problem entirely via static analysis, although with hindsight they could have done (missing GitHub attestation).
A security company could set up a honeypot machine that installs new releases of everything automatically and have a separate machine scan its network traffic for suspicious outbound connections.
Worth noting this attack was caught because people noticed anomalous network traffic to a new endpoint. The 7-day delay doesn't just give scanners time, it gives the community time to notice weird behavior from early adopters who didn't have the delay set.
It's herd immunity, not personal protection. You benefit from the people who DO install immediately and raise the alarm
But wouldn't the type of people that notifes anomalous network activity be exactly the type of people who add a 7 day delay because they're security conscious?
I suspect most packages will keep a mix of people at 7 days and those with no limit. That being said, adding jitter by default would be good to these features.
This became evident, what, perhaps a few years ago? Probably since childhood for some users here but just wondering what the holdup is. Lots of bad press could be avoided, or at least a little.
The config for uv won't work. uv only supports a full timestamp for this config, and no rolling window day option afaik. Am I crazy or is this llm slop?
> Define a dependency cooldown by specifying a duration instead of an absolute value. Either a "friendly" duration (e.g., 24 hours, 1 week, 30 days) or an ISO 8601 duration (e.g., PT24H, P7D, P30D) can be used.
This is what tripped me up. I added that config and then got this error:
error: Failed to parse: `.config/uv/uv.toml`
Caused by: TOML parse error at line 1, column 17
|
1 | exclude-newer = "7 days"
| ^^^^^^^^
failed to parse year in date "7 days": failed to parse "7 da" as year (a four digit integer): invalid digit, expected 0-9 but got
I was on version 0.7.20, so I removed that line, ran "uv self update" and upgraded to 0.11.2 and then re-added the config and it works fine now.
Yeah, that error message isn’t ideal on older versions, but unfortunately there’s no way to really address that. But I’m glad it’s working for you on newer versions.
I think it should work at the user config level too:
> If project-, user-, and system-level configuration files are found, the settings will be merged, with project-level configuration taking precedence over the user-level configuration, and user-level configuration taking precedence over the system-level configuration.
"Batteries included" ecosystems are the only persistent solution to the package manager problem.
If your first party tooling contains all the functionality you typically need, it's possible you can be productive with zero 3rd party dependencies. In practice you will tend to have a few, but you won't be vendoring out critical things like HTTP, TCP, JSON, string sanitation, cryptography. These are beacons for attackers. Everything depends on this stuff so the motivation for attacking these common surfaces is high.
I can literally count on one hand the number of 3rd party dependencies I've used in the last year. Dapper is the only regular thing I can come up with. Sometimes ScottPlot. Both of my SQL providers (MSSQL and SQLite) are first party as well. This is a major reason why they're the only sql providers I use.
Maybe I am just so traumatized from compliance and auditing in regulated software business, but this feels like a happier way to build software too. My tools tend to stay right where I left them the previous day. I don't have to worry about my hammer or screw drivers stealing all my bitcoin in the middle of the night.
> In practice you will tend to have a few, but you won't be vendoring out critical things like HTTP, TCP, JSON, string sanitation, cryptography
Unless you are Python, where the standard library includes multiple HTTP libraries and everyone installs the requests package anyways.
Few languages have good models for evolving their standard library, so you end up with lots of bad designs sticking around forever. Libraries are much easier to evolve, giving them the advantage in terms of developer UX and performance.
What type of developer chooses UX and performance over security? So reckless.
I removed the locks from all the doors, now entering/exiting is 87% faster!
After removing all the safety equipment, our vehicles have significantly improved in mileage, acceleration and top speed!
>What type of developer chooses UX and performance over security? So reckless.
Initially I assumed this is sarcastic, but apparently not. UX and performance is what programmers are paid to do! Making sure UX is good is one of the most important things in programmer job.
While security is a moving target, a goal, something that can never be perfect, just "good enough" (if NSA wants to hack you, they will). You make it sound like installing third party packages is basically equivalent to a security hole, while in practice the risk is low, especially if you don't overdo it.
Wild to read extreme security views like that, while at the same time there are people here that run unconstrained AI agents with --dangerous-skip-confirm flags and see nothing wrong with it.
Better developer UX can directly lead to better safety. "You are holding it wrong" is a frequent source of security bugs, and better UX reduces the ways you can hold it wrong, or at least makes you more likely to hold it the right way
The other thing that keeps coming up is the github-code-is-fine-but-the-release-artifact-is-a-trojan issue. It really makes me question if "packages" should even exist in JavaScript, or if we could just be importing standard plain source code from a git repo.
I understand why this doesn't work well with legacy projects, but it's something that the language could strive towards.
> I understand why this doesn't work well with legacy projects, but it's something that the language could strive towards.
Why wouldn't that work well with legacy projects? In fact, the projects I was a part of that I'd call legacy nowadays, was in fact built by copy-and-pasting .js libraries into a "vendor/" directory, and that's how we shipped it as well, this was in the days before Bower (which was the npm of frontend development back in the day), vendoring JS libs was standard practice, before package managers became used in frontend development too.
Not sure why it wouldn't work, JavaScript is a very moldable language, you can make most things work one way or another :)(
This is a rather superlative and tunnel vision, "everything is a nail because I'm a hammer" approach. The truth is this is an exceedingly difficult problem nobody has adequately solved yet.
I think the AI tooling is, if not completely solving sandboxing, at least making the default much better by asking you every time they want to do something and providing files to auto-approve certain actions.
Fully agree with this! I think today .NET is probably the most batteries included platform you can get. This means that even if you use third-party libraries, these typically depend only on first-party dependencies, making it much less likely for something shady to sneak in.
C#'s LINQ (code as data, like LISP) wins over golang for any type of data access. Strongly-typed, language-native queries. Go has its own advantages though.
What are some examples of batteries-included languages that folk around here really feel productive in and/or love? What makes them so great, in your opinion?
(Leaving aside thoughts on language syntax, compile times, tooling etc - just interested in people's experiences with / thoughts on healthy stdlibs)
These are the big ones I use, specifically because of the standard libraries:
Python (decent standard library) - It's pretty much everywhere. There's so many hidden gems in that standard library (difflib, argparse, shlex, subprocess, cmd)
C#/F# (.NET)
C# feels so productive because of how much is available in .NET Core, and F# gets to tag along and get it all for free too. With C# you can compile executables down to bundle the runtime and strip it down so your executables are in the 15 MiB range. If you have dotnet installed, you can run F# as scripts.
Do you worry at all about the future of F#? I've been told it's feeling more and more like a second-class citizen on .NET, but I don't have much personal experience.
For a lot of code, I switched to generating code rather than using 3rd party libraries.
Things like PEG parsers, path finding algorithms, string sanitizers, data type conversion, etc are very conveniently generated by LLMs. It's fast, reduces dependencies, and feels safer to me.
Or find the best third party library and copy the code from a widely used version that has been out long enough to have been well tested into your source tree.
The problem is not third party libraries. It is updating third party libraries when the version you have still works fine for your needs.
Things like unsafe string handling and data type conversion are some of the richest veins of security vulnerabilities I could possibly think of, and you're letting an LLM replace battle-handed library code because it "feels safer".
I admire your confidence in your ability to proof-read reams of "tedious" code, and come up with comprehensive tests for obscure edge cases. The kind of edge cases solved in established libraries for decades, which is why the average coder no longer even thinks about them. The kinds of edge cases that pay a black hat's bills.
The only reason you feel safer is because `pnpm audit` keeps you up to date on CVEs that affect you, which is unsettling, whereas your LLM whispers in your ear: "You are absolutely correct. This is production-ready code. It's not just secure — it's the right solution for your business. It's correct, safe, and performant. Don't overthink it — ship it. *green tick emoji*"
I'm not against LLMs, but there are good uses for LLMs, and then there's horror stories like this.
Honestly, you can get pretty far with just Bun and a very small number of dependencies. It’s what I love most about Bun. But, I do agree with you generally. .NET is about as good as I’ve ever seen for being batteries included. I just hate the enterprisey culture that always seems to pervade .NET shops.
I agree about the culture. If I take my eye off the dev team for too long, I'll come back and we'll be using entity framework and a 20 page document about configuring code cleanup rules in visual studio.
There’s a recurrent pattern with these package compromises: the attacker exfiltrates credentials during an initial phase, then pivots to the next round of packages using those credentials. That’s how we saw them make the Trivy to LiteLLM leap (with a 5 day gap), and it’ll almost certainly be similar in this case.
The solution to this is twofold, and is already implemented in the primary ecosystems being targeted (Python and JS): packagers should use Trusted Publishing to eliminate the need for long lived release credentials, and downstreams should use cooldowns to give security researchers time to identify and quarantine attacks.
(Security is a moving target, and neither of these techniques is going to work indefinitely without new techniques added to the mix. But they would be effective against the current problems we’re seeing.)
In this case, the author's NPM account was taken over, email address changed to one the attacker controls, and the package was manually published.
Since the attacker had full control of the NPM account, it is game over - the attacker can login to NPM and could, if they wanted, configure Trusted Publishing on any repo they control.
Axios IS using trusted publishing, but that didn't do anything to prevent the attack since the entire NPM account was taken over and config can be modified to allow publishing using a token.
I don’t know what this means. Discipline is good, but I think you need to have good tools/primitives in place to help people exercise discipline.
(The classic example being passwords: we wouldn’t need MFA is everybody just “got good” and used strong/unique passwords everywhere. But that’s manifestly unrealistic, so instead we use our discipline budget on getting people to use password managers and phishing-resistant MFA.)
I can't even imagine the scale of the impact with Axios being compromised, nearly every other project uses it for some reason instead of fetch (I never understood why).
Also from the report:
> Neither malicious version contains a single line of malicious code inside axios itself. Instead, both inject a fake dependency, plain-crypto-js@4.2.1, a package that is never imported anywhere in the axios source, whose only purpose is to run a postinstall script that deploys a cross-platform remote access trojan (RAT)
Good news for pnpm/bun users who have to manually approve postinstall scripts.
> nearly every other project uses it for some reason instead of fetch (I never understood why).
Fetch wasn't added to Node.js as a core package until version 18, and wasn't considered stable until version 21. Axios has been around much longer and was made part of popular frameworks and tutorials, which helps continue to propagate it's usage.
There are pretty much two usage patterns that come up all the time:
1- automatically add bearer tokens to requests rather than manually specifying them every single time
2- automatically dispatch some event or function when a 401 response is returned to clear the stale user session and return them to a login page.
There's no reason to repeat this logic in every single place you make an API call.
Likewise, every response I get is JSON. There's no reason to manually unwrap the response into JSON every time.
Finally, there's some nice mocking utilities for axios for unit testing different responses and error codes.
You're either going to copy/paste code everywhere, or you will write your own helper functions and never touch fetch directly. Axios... just works. No need to reinvent anything, and there's a ton of other handy features the GP mentioned as well you may or may not find yourself needing.
That fetch requires so many users to rewrite the same code - that was already handled well by every existing node HTTP client- says something about the standards process.
It could also be trivially written for XMLHttpRequest or any node client if needed. Would be nice if they had always been the same, but oh well - having a server and client version isn't that bad.
Because it is so few lines it is much more sensible to have everyone duplicate that little snippet manually than import a library and write interceptors for that...
(Not only because the integration with the library would likely be more lines of code, but also because a library is a significantly liability on several levels that must be justified by significant, not minor, recurring savings.)
Don't be silly, this is the JS ecosystem. Why use your brain for a minute and come up with a 50 byte helper function, if you can instead import a library with 3912726 dependencies and let the compiler spend 90 seconds on every build to tree shake 3912723 out again and give you a highly optimized bundle that's only 3 megabytes small?
IMO interceptors are bad. they hide what might get transformed with the API call at the place it is being used.
> Likewise, every response I get is JSON. There's no reason to manually unwrap the response into JSON every time.
This is not true unless you are not interfacing with your own backends. even then why not just make a helper that unwraps as json by default but can be passed an arg to parse as something else
One more use case for Axios is it automatically follows redirects, forwarding headers, and more importantly, omiting or rewriting the headers that shouldn't be forwarded for security reasons.
> I can't even imagine the scale of the impact with Axios being compromised, nearly every other project uses it for some reason instead of fetch (I never understood why).
You can remember this answer for every time you ask same question again:
"Coz whatever else/builtin was before was annoying enough for common use cases"
Because axios existed before the builtin fetch, and so there's a lot of stackoverflow answers explaining how to use fetch, and the llm models are trained on that, so they will write axios requests instead of fetch
> Good news for pnpm/bun users who have to manually approve postinstall scripts.
Would they not have approved it for earlier versions? But also wouldn't the chance of addition automatic approval be high (for such a widely used project)?
The prompt would be to approve the new malicious package (plain-crypto-js)'s scripts, too, which could tip users off that something was fishy. If they were used to approving one for axios and the attackers had just overwrote axios's own instead of making a new package, it would probably catch people out.
Assuming axios didn't have a postinstall script before, it wouldn't have been approved for a previous version. If you ignore it, you ignore it, but postinstall scripts are relatively rare in npm deps, so it would seem a bit out of place when the warning pops up.
Can't speak for other devs but I like to read postinstall scripts or at least put them through an LLM if they're too hard to grok.
It's also a little context dependent, for example if I was using Axios and I see a prompt to run the plain-crypto-js postinstall script, alarm bells would instantly ring, which would at least make me look up the changelog to see why this is happening.
In most cases I don't even let them run unless something breaks/doesn't work as expected.
Does pnpm block postinstall on transitive deps too or just top-level? We have it configured at work but I've never actually tested whether it catches scripts from packages that get pulled in as sub-dependencies.
It prompts for transitive dependencies, too. I have never had workerd as a direct dependency of any project of mine but I get prompted to approve its postinstall script whenever I install cloudflare's wrangler package (since workerd needs to download the appropriate Workers runtime for your platform).
That's solid, really helps lock down the supply chain attack surface. Do you ever end up having to whitelist anything that legitimately needs to run on install?
After using pnpm for years (at least 5, don't remember exactly), I've only ever had to whitelist one library that uses a postinstall script to download a native executable for your system. And even this is not necessary, it's just poorly designed.
For example, esbuild and typescript 7 split binaries for different systems and architectures into separate packages, and rely on your package manager to pull the correct one.
Setting min-release age to 7 days is great, but the only true way to protect from supply chain attacks is restricting network access.
This needs to be done (as we've seen from these recent attacks) in your devenv, ci/cd and prod environments. Not one, or two, but all of these environments.
The easiest way is via using something like kubernetes network policies + a squid proxy to allow limited trusted domains through, and those domains must not be publicly controllable by attackers. ie. github.com is not safe to allow, but raw.githubusercontent.com would be as it doesn't allow data to be submitted to it.
This stops both DNS exfil and HTTP exfil. For your devenv, software like Little Snitch may protect your from these (I'm not 100% on DNS exfil here though). Otherwise run your devenv (ie vscode) as a web server, or containerised + vnc, a VM, etc, with the same restrictions.
Not to beat a dead horse but I see this again and again with dependencies. Each time I get more worried that the same will happen with rust. I understand the fat std library approach won’t work but I really still want a good solution where I can trust packages to be safe and high quality.
Linux distros and BSD ports did that since the 90's. When Linux distros had barely a PM or just tarballs, Infomagic sold 4 CD full of libre software. When I had no internet at home, back in the day I bought 3 DVD's of Debian Sarge for 20 euros, about $20. A bargain, it was the price of a hard-cover best seller book.
GB's of libre software, graphical install, 2.6 kernel, KDE3 desktop, very light on my Athlon 2000 with 256MB of RAM. It was incredible compared to what you got with Windows XP and 120 Euro per seat. Nonfree software and almost empty.
And, well, if for instance I could get read only, ~16TB durable USB drive with tons of Guix packages offline (in a two yearly basis with stable releases) for $200 I would buy them in the spot.
You would say that $200 for a distro it's expensive, but for what it provides, if you are only interested in libre gaming and tools, they amount you save can be huge. I've seen people spend $400 in Steam games because of the Holyday sales...
Why wouldn't the "fat std" thing work? Yes it's hard to design properly, both in scope and actual design (especially for an unstandardized language still moving fast), but throwing the towel and punting the problem to the "free market" of uncurated public repos is even worse.
It's what we call in France "la fête du slip".
PS: that's one reason I try to use git submodules in my Common Lisp projects instead of QuickLisp, because I really see the size of my deptree this way.
Not necessarily, but let's agree that some design faults would happen: you still get the option to use the solid, boring and slightly rusty std instead of another 100 dependencies from the supply chain supermarket.
At work, we're happy with Python's included batteries when we need to make scripts instead of large programs.
NPM should have a curation mechanism, via staff review or crowdsourcing, where versions of popular packages are promoted to a stable set, like linux distros do. I would only use curated versions if they had such a thing.
- copy the dependencies' tests into your own tests
- copy the code in to your codebase as a library using the same review process you would for code from your own team
- treat updates to the library in the same way you would for updates to your own code
Apparently, this extra work will now not be a problem, because we have AI making us 10x more efficient. To be honest, even without AI, we should've been doing this from the start, even if I understand why we haven't. The excuses are starting to wear thin though.
I don't know where you've worked but a hostile and intelligent actor or internal red team would succeed under each of those cases at every job I've worked at.
Defending against a targeted attack is difficult, yes. But these recent campaigns were all directed at everyone. Auditing and inspecting your dependencies does absolutely help thwart that because there will always be people who don't.
I recommend everyone to use bwrap if you're on linux and alias all package managers / anything that has post build logic with it.
I have bwrap configured to override: npm, pip, cargo, mvn, gradle, everything you can think of and I only give it the access it needs, strip anything that is useless to it anyway, deny dbus, sockets, everything. SSH is forwarded via socket (ssh-add).
This limits the blast radius to your CWD and package manager caches and often won't even work since the malware usually expects some things to be available which are not in a permissionless sandbox.
You can think of it as running a docker container, but without the requirement of having to have an image. It is the same thing flatpak is based on.
As for server deployments, container hardening is your friend. Most supply chain attacks target build scripts so as long as you treat your CI/CD as an untrusted environment you should be good - there's quite a few resources on this so won't go into detail.
This only works for post-install script attacks. When the package is compromised, just running require somewhere in your code will be enough, and that runs with node/java/python and no bwrap.
node is also sandboxed within bwrap I have sandbox -p node if I have to give node access to other folders, I also have sandbox -m to define custom mountpoints if necessary and UNSAFE=1 as a last resort which just runs unsandboxed.
Check also https://github.com/wrr/drop which is a higher-level tool than bwrap. It allows you to make such isolated sandboxes with minimal configuration.
This looks nice but I wouldn't trust a very fresh tool to do security correctly.
As a higher-level alternative to bwrap, I sometimes use `flatpak run --filesystem=$PWD --command=bash org.freedesktop.Platform`. This is kind of an abuse of flatpaks but works just fine to make a sandbox. And unlike bwrap, it has sane defaults (no extra permissions, not even network, though it does allow xdg-desktop-portal).
I like the idea of bubblewrap, but my pain point is that it is work to set it up correctly with bind mounts and forwarding necessary environment variables to make the program actually work usefully. Could you share your pip bwrap configuration? It sounds useful.
AFAIK maven doesn’t support post install logic like npm does. You have to explicitly optin with build plugins. It doesn’t let any arbitrary dependency run code on your machine.
Maybe I misunderstood this point. But the ssh socket also gives access to your private keys, so I see no security gain in that point. Better to have a password protected key.
It's so your private key is not stolen, but you're right passphrase protected keys win anyway. I use hardware keys so this isn't a problem for me to begin with.
We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a ton of incredibly useful, non-trivial work for you, while barely increasing your executable's size.
The issue is not dependencies themselves, it's transitive ones. Nobody installs left-pad or is-even-number directly, and "libraries" like these are the vast majority of the attack surface. If you get rid of transitive dependencies, you get rid of the need of a package manager, as installing a package becomes unzipping a few files into a vendor/ folder.
There's so many C libraries like this. Off the top of my head, SQLite, FreeType, OpenSSL, libcurl, libpng/jpeg, stb everything, zlib, lua, SDL, GLFW... I do game development so I'm most familiar with the ones commonly used in game engines, but I'm sure other fields have similarly high quality C libraries.
They also bindings for every language under the sun. Rust libraries are very rarely used outside of Rust, and C#/Java/JS/Python libraries are never used outside their respective language (aside form Java ones in other JVM langs).
They're not a failed experiment. No one has ever "experimented" by making a safe package manager for their new language. And it is not that insane to do so. Very basic things will get you very far:
1. Packages should carry a manifest that declares what they do at build time, just like Chrome extensions do. This manifest would then be used to configure its build environment.
2. Publishers to official registries should be forced to use 2FA. I proposed this a decade ago for crates.io and people lost their minds, like I was suggesting we drag developers to a shed to be shot.
3. Every package registry should produce a detailed audit log that contains a "who, what, when". Every build/ command should be producing audit logs that can be collected by endpoint agents too.
4. Every package registry should support TUF.
5. Typosquatting defenses should be standard.
etc etc etc. Some of this is hard, some of this is not hard. All of this is possible. No one has done it, so it's way too early to say "package managers can't be made safe" when no one has tried.
Package managers are now basically a requirement for language adoption. Doing it manually is not a solution, in an automated world.
What is a problem is library quality. Which is downstream of nobody getting paid for it, combined with an optimistic but unrealistic "all packages are equal" philosophy.
> High quality C libraries
> OpenSSL
OpenSSL is one of the ones where there's a ground up rewrite happening because the code quality is so terrible while being security critical.
On the other end, javascript is uniquely bad because of the deployment model and difficulty of adding things to the standard library, so everything is littered with polyfills.
> Package managers are now basically a requirement for language adoption. Doing it manually is not a solution, in an automated world.
Absolute nonsense. What does automated world even mean? Even if one could infer reasonably, it's no justification. Appealing to "the real world" in lieu of any further consideration is exactly the kind of mindlessness that has led to the present state of affairs.
Automation of dependency versions was never something we needed it was always a convenience, and even that's a stretch given that dependency hell is abundant in all of these systems, and now we have supply chain attacks. While everyone is welcome to do as they please, I'm going to stick to vendoring my dependencies, statically compiling, and not blindly trusting code I haven't seen before.
Relax, while mentioning the real world without any criticism for the soundness of the solution is absolute nonsense, some would say idiotic, thinking only in the absolute best solution given your narrow world view is not any better.
While I agree that my view is narrow, the "best solution" in question is what we used to do, and it was fine. There are still many places that manually manage dependencies. Fundamentally automatic software versioning is an under-developed area in need of attention, and technologies like semantic versioning which are ubiquitous are closer to suggestions, and not true indicators of breaking changes. My personal view is that fully automatic dependency version management is an ongoing experiment and should be treated as such.
> We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a ton of incredibly useful, non-trivial work for you, while barely increasing your executable's size.
I'm not sure why you believe this is more secure than a package manager. At least with a package manager there is an opportunity for vetting. It's also trivial that it did not increase your executable's size. If your executable depends on it, it increases its effective size.
If you're developing for the web your attack surface is quite a bit bigger. Your proposed solution of copying a few files might work but how do you keep track of updates? You might be vulnerable to a published exploit fixed a few months ago. A package manager might tell you a new version is available. I don't know how that would work in your scenario.
I don't think this community of professionals is going to come around to a solution which requires marginally more effort.
If no one checks their dependencies, the solution is to centralize this responsibility at the package repository. Something like left-pad should simply not be admitted to npm. Enforce a set of stricter rules which only allow non-trivial packages maintained by someone who is clearly accountable.
Another change one could make is develop bigger standard libraries with all the utilities which are useful.
For example in Rust there are a few de facto standard packages one needs very often, which then also force you to pull in a bunch of transitive dependencies.
Those could also be part of the standard library.
This all amounts to increasing the minimal scope of useful functionality a package has to have to be admitted and increasing accountability of the people maintaining them. This obviously comes with more effort on the maintainers part, but hey maybe we could even pay them for their labor.
The culture within the npm/js community has mainly been one of using the package manager rather than "re-inventing the wheel", as such the blast radius of a compromised package is much greater
It's more to do with the standard library being so barren of common application needs, and looking for a solution that the community has gotten behind. Axios has been a common dependency in many codebases, because it is a solid solution that many have already used. Every developer could try building all the libraries that they would reach for themselves, but then each company has now taken on the task of ensuring their own (much larger) codebase is free from security issues, on top of taking care of their own issues and bugs.
It’s not just NPM, though. Every Rails project and every Rust project I’ve seen ended up with massive numbers of dependencies vs what an equivalent project in Go or C# would have needed.
CPAN too, just try Hailo under Perl to test an old-fashioned chatbot based on Markov chains where very small LLM's and Hailo converge if used with the advanced training options for it. Yes, it will pull tons of dependencies, (less with cpanminus if run with 'cpanm -n Hailo'), but contrary to NPM, Pip and the like CPAN's repos are highly curated and before PHP and ubiquitoous Python Perl was used everywhere, from a sysadmin language (better than Bash/Sh for sure) to CGI, IRC bots and whatnot. How many issues did we have? Zero or near zero.
Rust libraries are infrequently used outside of Rust because if you have the option, you'd just use Rust, not the ancient featureless language intrinsically responsible for 70% of all security issues. C libraries are infrequently used in Rust outside of system libc, for the same reason; I go and toggle the reqwest switch to use rustls every time, because OpenSSL is horrendous. This is also why you say 'rarely' instead of 'never', when a few years ago it was 'never'; a few years from now you'll say 'uncommonly', and so on. The reason C libraries are used is because you don't feel like reimplementing it yourself, and they are there; but that doesn't apply more to C libraries than Rust libraries, and the vast majority of crates.io wouldn't be usefully represented in C anyway, or would take longer to bind to than to rewrite. (No, nobody uses libcurl.) Finally, this only happens in NPM, and the Rust libraries you pull in are all high-quality. So this sounds like a bunch of handwaving about nonsense.
Rust is terrible for pulling in hundreds of dependencies though. Add tokio as a dependency and you'll get well over 100 packages added to your project.
pin-project-lite is the only base dependency, which itself has no dependencies. If you enable the "full" feature, ie all optional doodads turned on (which you likely don't need), it's 17: bytes, cfg-if, errno, libc, mio, parking_lot+parking_lot_core+lock_api, pin-project-lite, proc_macro2+quote+syn+unicode-ident, scopeguard, signal-hook-registry, smallvec, and socket2. You let me know which ones you think are bloat that it should reimplement or bind to a C library about, and without the blatant fabrication this time.
I'd really like to see package managers organized around rings where a very small core of incredibly important stuff is kept in ring 0, ring 1 gets a slightly wider amount of stuff and can only depend on ring 0 dependencies and then ring 2+ is the crapware libraries that infect most ecosystems.
But maybe that's not the right fit either. The world where package managers are just open to whatever needs to die. It's no longer a safe model.
The OS distro model is actually the right one here. Upstream authors hate it, but having a layer that's responsible for picking versions out of the ecosystem and compiling an internally consistent grouping of known mutually-compatible versions that you can subscribe to means that a lot of the random churn just falls away. Once you've got that layer, you only need to be aware of security problems in the specific versions you care about, you can specifically patch only them, and you've got a distribution channel for the fixes where it's far more feasible to say "just auto-apply anything that comes via this route".
That model effectively becomes your ring 1. Ring 0 is the stdlib and the package manager itself, and - because you would always need to be able to step outside the distribution for either freshness or "that's not been picked up by the distro yet" reasons - the ecosystem package repositories are the wild west ring 2.
In the language ecosystems I'm only aware of Quicklisp/Ultralisp and Haskell's Stackage that work like this. Everything else is effectively a rolling distro that hasn't realised that's what it is yet.
In practice, "ring 0" is whatever gets merged into your language's standard library. Node and python both have pretty expansive standard libraries at this point, stepping outside of those is a choice
> We have libraries like SQLite, which is a single .c file that you drag into your project
You are just swapping a package manager with security by obscurity by copy pasting code into your project. It is arguably a much worse way of handling supply chain security, as now there is no way to audit your dependencies.
> If you get rid of transitive dependencies, you get rid of the need of a package manager
This argument makes no sense. Obviously reducing the amount of transitive dependencies is almost always a good thing, but it doesn't change the fundamental benefits of a package manager.
> There's so many C libraries like this
The language with the most fundamental and dangerous ways of handling memory, the language that is constantly in the news for numerous security problems even in massively popular libraries such as OpenSSL? Yes, definitely copy-paste that code in, surely nothing can go wrong.
> They also bindings for every language under the sun. Rust libraries are very rarely used outside of Rust
This is a WILD assumption, doing C-style bindings is actually quite common. YOu will of course then also be exposing a memory unsafe interface, as that is what you get with C.
What exactly is your argument here? It feels like what you are trying to say is that we should just stop doing JS and instead all make C programs that copy paste massive libraries because that is somhow 'high quality'.
This seems like a massively uninformed, one-sided and frankly ridiculous take.
> You are just swapping a package manager with security by obscurity by copy pasting code into your project
You should try writing code, and not relying on libraries for everything, it may change how you look at programming and actually ground your opinions in reality. I'm staring at company's vendor/ folder. It has ~15 libraries, all but one of which operate on trusted input (game assets).
> fundamental benefits of a package manager.
I literally told you why they don't matter if you write code in a sane way.
> doing C-style bindings is actually quite common
I know bindings for Rust libraries exist. Read the literal words you quoted. "Rust libraries are very rarely used outside of Rust". Got some counterexamples?
Package managers are older than some users here. From CPAN/CTAN to ports under BSD's.
Some pm's are badly maintained (Pip/NPM), while others are curated enough.
Again, if you have GNU/Linux installed, install Guix, read the Info manual on 'guix import' and just create a shell/container with 'guix shell --container' (and a manifest package created from guix import) and use any crap you need for NPM in a reproducible and isolated way. You $HOME will be safe, for sure.
I think you can do copy paste in most languages. But it will be a pain to update when there are improvements / security fixes.
You got a project with 1-2 depencies? Sure. But if you need to bring in 100 different libs (because you bring in 10 libs which in turn brings in 10 libs) good luck.
> But if you need to bring in 100 different libs (because you bring in 10 libs which in turn brings in 10 libs
So don’t?
With manual deps management, everyone soon gravitates to a core set of deps. And libraries developer tends to reduce their deps needs, That’s why you see most C libraries deals with file formats, protocols, and broad concerns. Smaller algorithms can be shared with gists and blog articles.
Genuinely how are you supposed to make sure that none of the software you have on your system pulls this in?
It’s things like this that make me want to swap to Qubes permanently, simply as to not have my password manager in the same context as compiling software ever.
While it's not perfect, pinning specific versions and managing all updates directly has been a solid solution for my team. Things can of course still slip through, but we're never vulnerable to these just because there was a new package release and we opted into it by default.
Updating packages takes longer, but we try to keep packages to a minimum so it ends up not being that big deal.
We run everything NPM related inside Apple containers, and are looking to do the same with Python and Rust soon. Bwrap on Linux does the same.
I like to think of it like working with dangerous chemicals in the lab. Back in the days, people were sloppy and eventually got cancer. Then dangers were recognized and PPE was developed and became a requirement.
We are now at the stage in software development where we are beginning to recognizing the hazards and developing + mandating use of proper PPE.
A couple of years ago, pip started refusing to install packages outside of a virtualenv. I'm guessing/hoping package managers will start to have an opt-in flag you can set in a system-wide config file, such that they refuse to run outside of a sandbox.
The problem is that package managers are a distraction. You have to sandbox everything or else it doesn't work. These attacks use post-install hooks for convenience but nothing would have stopped them patching axios itself and just waiting for devs to run the app on their local workstation. So you end up needing to develop in a fully sandboxed environment.
This sounds like satire but isn't - I just make sure the nodejs/npm packages don't exist on my system. I've yet to find a crucial piece of software that requires it. As much as I love that cute utility that turns maps into ascii art, it's not exactly sqlite in terms of usefulness.
I don't deny that node/npm is useful for building servers, devtools for JS development itself, etc. but as an end user I haven't encountered anything useful which requires having it on my machine.
How much do you want to bet me that the credential was stolen during the previous LiteLLM incident? At what point are we going to have to stop using these package managers because it's not secure? I've got to admit, it's got me nervous to use Python or Node.js these days, but it's really a universal problem.
> it’s got me nervous to use Python or Node.js these days
My feelings precisely. Min package age (supported in uv and all JS package managers) is nice but I still feel extremely hesitant to upgrade my deps or start a new project at the moment.
I don’t think this is going to stabilize any time soon, so figuring out how to handle potentially compromised deps is something we will all need to think about.
I would be avoiding npm itself on principle in the JS ecosystem. Use a package manager that has a history of actually caring about these issues in a timely manner.
PNPM makes you approve postinstall scripts instead of running them by default, which helps a lot. Whenever I see a prompt to run a postinstall script, unless I know the package normally has one & what it does, I go look it up before approving it.
(Of course I could still get bitten if one of the packages I trust has its postinstall script replaced.)
There's a package manager discussion, but the bit that stands out to me is that this started with a credential compromise. At some point when a project gets big enough like axios, maybe the community could chip in to buy the authors a couple of YubiHSM or similar. I wish that _important keys live in hardware_ becomes more standard given the stakes.
Dealing with dependencies is another question; if it's stupid stuff like leftpad then it should be either vendored in or promoted to be a language feature anyway (as it has been).
> At some point when a project gets big enough like axios, maybe the community could chip in to buy the authors a couple of YubiHSM or similar
I kind of feel like the authors here should want that for themselves, before the community would even realize it's needed. I can't say I've worked on packages that are as popular as axios, but once some packages we were publishing hit 10K downloads or so, we all agreed that we needed to up our security posture, and we all got hardware keys for 2FA and spent 1-2 weeks on making sure it was as bullet-proof we could make it.
To be fair, most FOSS is developed by volunteers so I understand not wanting to spend any money on something you provide for free, but on the other hand, I personally wouldn't feel comfortable being responsible for something that popular without hardening my own setup as much as I could, even if it means stopping everything for a week.
Also, considering how prevalent TPM/Secure Enclaves are on modern devices, I would guess most package maintainers already have hardware capable of generating/using signing keys that never leave hardware.
I think it is mostly a devex/workflow question.
Considering the recent ci/cd-pipeline compromises, I think it would make sense to make a two phase commit process required for popular packages. Build and upload to the registry from a pipeline, but require a signature from a hardware resident key before making the package available.
Most of axios' functionality has effectively been promoted to a language feature as `fetch`, but the problem is people don't bother to migrate. I've migrated our direct usage of it but it's still pulled in transitively in several parts of our codebase.
Even left-pad is still getting 1.6 million weekly downloads.
Annoyingly, the times I reach for axios and similar is when I need to keep track of upload progress, which I could only do with XMLHttpRequest, not fetch, unless I've missed some recent browser changes, and the API of XMLHttpRequest remains as poor as the first times I had to use it. Download progress been supported by fetch since you can track chunks yourself, but somehow they didn't think to do that for requests for some reason, only responses.
>maybe the community could chip in to buy the authors a couple of YubiHSM
There's no community, the users of axios are devs that looked at stackoverflow for "how to download a file in javascript", they barely know or care what axios is.
Now the users of axios are devs that ask Claude Code or Codex to scrape a website or make a dashboard, they don't even know about the word axios.
I personally had to delete axios a couple of time from my codebase when working with junior devs.
Or those people can (fund) separate repackaging and redistribution with more stringent and formalized review process.
Maybe not all users should pull all packages straight from what devs are pushing.
There's no reason we can't have "node package distributions" like we have Linux distributions. Maybe we should stop expecting devs and maintainers and Microsoft to take responsibility for our supply-chain.
Essential steps to minimise your exposure to NPM supply chain attacks:
— Run Yarn in zero-installs mode (or equivalent for your package manager). Every new or changed dependency gets checked in.
— Disable post-install scripts. If you don’t, at least make sure your package manager prompts for scripts during install, in which case you stop and look at what it’s going to run.
— If third-party code runs in development, including post-install scripts, try your best to make sure it happens in a VM/container.
— Vet every package you add. Popularity is a plus, recent commit time is a minus: if you have this but not that, keep your eyes peeled. Skim through the code on NPM (they will probably never stop labelling it as “beta”), commit history and changelog.
— Vet its dependency tree. Dependencies is a vector for attack on you and your users, and any new developer in the tree is another person you’re trusting to not be malicious and to take all of the above measures, too.
Number 1 would only be a win for zero-installs if it happened that registry was up when you made the security hotfix, since you'd need to install the depdencency the first time to get it in VC, but then suddenly down when doing a deploy. Seems like a highly unlikely scenario to me. Also, cases where npm CVEs must be patched with such urgency or bad things will happen are luckily very rare, in my experience.
Most npm CVEs are stuff like DDoS vulnerabilities, and you should have mitigations for those in place for at the infra-level anyway (e.g. request timeouts, rate limits, etc), or you are pretty much guaranteed to be cooked sooner or later anyway. The really dangerous stuff like arbitrary command execution from a library that takes end user input is much much more rare. The most recent big one I remember is React2shell.
Number 2 hasn't been much of an issue for a long time. npm doesn't allow unpublishing package after 72 hours (apart from under certain rare conditions).
Don't know about number 3. Would feel to me that if you have something running that can modify lockfile, they can probably also modify the chekced-in tars.
I can see how zero-installs are useful under some specific constraints where you want to minimize dependencies to external services, e.g. when your CI runs under strict firewalls. But for most, nah, not worth it.
> you'd need to install the depdencency the first time to get it in VC, but then suddenly down when doing a deploy.
Which dependency? It sounds like you are assuming some specific scenario, whereas the fix can take many forms. In immediate term, the quickest step could be to simply disable some feature. A later step may be vendoring in a safe implementation.
The registry doesn’t need to be actually down for you, either; the necessary condition is that your CI infrastructure can’t reach it.
> cases where npm CVEs must be patched with such urgency or bad things will happen are luckily very rare, in my experience.
Not sure what you mean by “npm CVEs”. The registry? The CLI tool?
As I wrote, if you are running compromised software in production, you want to fix it ASAP. In first moments you may not even know whether bad things will happen or not, just that you are shipping malicious code to your users. Even if you are lucky enough to determine with 100% confidence (putting your job on the line) that the compromise is inconsequential, you don’t want to keep shipping that code for another hour because your install step fails due to a random CI infra hiccup making registry inaccessible (as happened in my experience at least half dozen times in years prior, though luckily not in a circumstance where someone attempted to push an urgent security fix). Now imagine it’s not a random hiccup but part of a coordinated targeted attack, and somehow it becomes something anticipated.
> Number 2 hasn't been much of an issue for a long time. npm doesn't allow unpublishing package after 72 hours (apart from under certain rare conditions).
Those rare conditions exist. Also, you are making it sound as if the registry is infallible, and no humans and/or LLMs there accept untrusted input from their environment.
The key aspect of modern package managers, when used correctly, is that even when the registry is compromised you are fine as long as integrity check crypto holds up and you hold on to your pre-compromise dependency tree. The latter is not a technical problem but a human problem, because conditions can be engineered in which something may slip past your eyes. If this slip-up can be avoided at little to no cost—in fact, with benefits, since zero-installs shortens CI times, and therefore time-to-fix, due to dramatically shorter or fully eliminated install step—it should be a complete no-brainer.
> Don't know about number 3. Would feel to me that if you have something running that can modify lockfile, they can probably also modify the chekced-in tars.
As I wrote, I suspect it’d complicate such attacks or make them easier to spot, not make them impossible.
Exactly. Yarn uses a yarn.lock file with the sha256 hashes of each npm package it downloads from the repo (they are .tgz files). If the hash won't match, install fails. No need to commit the dependencies into your git.
I know there is a cooldown period for npm packages, but I’m beginning to want a cooldown for domains too. According to socket, the C2 server is sfrclak[.]com, which was registered in the last 24 hours.
This may not be popular, but is there a place for required human actions or just timed actions to slow down things like this? For instance, maybe a GH action to deploy requires a final human click and to change that to cli has a 3 day cooling period with mandatory security emails sent out. Similarly, you switch to read only for 6 hrs after an email change. There are holes in these ideas but the basic concept is to treat security more like physical security, your goal isn't always to 100% block but instead to slow an attacker for xxx minutes to give the rest of the team time to figure out what is going on.
Hi, security here. We've tried, but the amount of people you need for this vs the amount of people you have trying to review and click the big button always means that this step will be a bottleneck. Thus this step will be eliminated.
A much better approach would be to pin the versions used and do intentional updates some time after release, say a sprint after.
Yeah, I am looking at that on the use end. It sounds like on the python side this type of thing will be more standard (uv now and soon pip supported with version date requirements). I think time is a big missing element in many security in depth decisions. It can be time until you adopt like use no package newer than xx days or time it takes to deploy etc etc. Unfortunately the ecosystem is getting really diverse and that means ever more sophisticated attacks so we may need to do things that are annoying just to survive.
Why not just release escrow? If I try to push a new release version another developer or developers have to agree to that release. In larger projects you would expect the release to be coordinated or scheduled anyways. Effectively we're just moving "version pinning" or "version delay" one layer up the release chain.
With all the recent supply chain attacks, I'm starting to think it's only a matter of time before all of us are victims. I think this is a sign to manually check all package diffs or postinstall scripts.
Supply chain attacks are so scary that I think most companies are going to use agents to hard fork their own versions of a lot of these core libraries instead. It wasn’t practical before. It’s definitely much more doable today.
I was thinking about this as a bull case for human developers. Seems if you're worried enough to do this you're not going to have LLMs write the new code.
There are so many scanners these days these things get caught pretty quick. I think we need either npm or someone else to have a registry that only lets through packages that pass these scanners. Can even do the virustotal thing of aggregating reports by multiple scanners. NPM publishes attestation for trusted build environments. Google has oss-rebuild.
All it takes is an `npm config set` to switch registries anyways. The hard part is having a central party that is able to convince all the various security companies to collaborate rather than having dozens of different registries each from each company.
Rather than just a hard-coded delay, I think having policies on what checks must pass first makes sense with overrides for when CVEs show up.
The ones you hear about are caught quickly, I’m more worried about the non obvious ones. So far none of these have been as simple as changing a true to a false and bypassing all auth for all products or something, and would that be caught by an automated scanner?
There are definitely levels to this. Yes I think it can be caught by automated scanners in theory. Either commit by commit scanning and reproducible builds or fuzzing and getting the behavioral differences between versions
The frustrating thing here is that axios versions display on npmjs with verified provenance. But they don’t use trusted publishing: https://github.com/axios/axios/issues/7055 - meaning the publish token can be stolen.
I wrongly thought that the verified provenance UI showed a package has a trusted publishing pipeline, but seems it’s orthogonal.
NPM really needs to move away from these secrets that can be stolen.
This is just going to get worse and worse as agentic coding gets better. I think having a big dependency tree may be a thing of the past in the coming years. Seems like eventually new malware will be coming out so fast it will basically be impossible to stop.
Ran npm ci --ignore-scripts in our CI for months but never thought about local dev. Turns out that's the gap, your CI is safe but your laptop runs postinstall on every npm install.
The anti-forensics here are much more complicated that I had imagined. Sahring after getting my hands burned.
After the RAT deploys, setup.js deletes itself and swaps package.json with a clean stub. Your node_modules looks fine. Only way to know is checking for artifacts: /Library/Caches/com.apple.act.mond on mac, %PROGRAMDATA%\wt.exe on windows, /tmp/ld.py on linux. Or grep network logs for sfrclak.com.
Somehow noboady is worried about how agentic coding tools run npm install autonomously. No human in the loop to notice a weird new transitive dep. That attack surface is just getting worsened day by day.
In case it helps, we open-sourced a tool to audit dependencies for this kind of supply-chain issue. The motivation was that there is a real gap between classic “known vulnerability” scanning and packages whose behavior has simply turned suspicious or malicious. We also use AI to analyze code and dependency changes for more novel or generic malicious behavior that traditional scanners often miss.
I'm working on a multi signature solution that helps to detect unauthorized releases in the case of an account hijack. It is open source, self hostable, accountless and I am looking for feedback!
I am glad I don't need to touch JS or web dev at all.
Now, I tend to use Python, Rust and Julia. With Python I am constantly using few same packages like numpy and matplotlib. With Rust and Julia, I try as much as possible to not use any packages at all, because it always scares me when something that should be pretty simple downloads half of the Internet to my PC.
Julia is even worse than Rust in that regard - for even rudimentary stuff like static arrays or properly namespaced enums people download 3rd party packages.
Isn't Rust just as susceptible to this issue? For example, how do you deal with Rust's lack of support for HTTP in the standard library? Importing hyper pulls in a couple dozen transitive libraries which exposes you to the exact same kind of threats that compromised axios.
Given how HTTP is now what TCP was during the 90s and almost all modern networked applications needing to communicate in it one way or another, most rust projects come with an inherent security risk.
These days, I score the usability of programming languages by how complete their standard library is. By that measure, Rust and Javascript get an automatic F.
It is, therefore I have stated I avoid any dependencies while writing Rust, unless they are self-contained. And I said I am glad I don't do web, so I don't have need for HTTP implementations.
All these supply chain attacks make me nervous about the apps I use. It would be valuable info if an app used such dependencies, but on the other hand, programmers would cut their sales if they gave you this info.
Min release age sucks, but we’ve been here before. Email attachments used to just run wild too, then everyone added quarantine delays and file blocking and other frictions... and it eventually kinda/sorta worked. This does feel worse, though, with fewer chokepoints and execution as a natural part of the expectation.
Edit: bottom line is installs are gonna get SOOO much more complicated. You can already see the solution surface... Cooling periods, maintainer profiling, sandbox detonation, lockfile diffing, weird publish path checks. All adds up to one giant PITA for fast easy dev.
Min release age might just postpone vulnerability to be applied few days later in non trivial cases like this. More I think about it, Odin lang approach of no package manager makes senses. But, for that approach won't work for Javascript as it needs npm package even for trivial things. Even vendoring approach like golang won't work with Javascript with the amount of churn and dependencies.
So the root cause was again a developer's opsec. For improving things, I haven't seen many new initiatives on that side (beyond 2FA, but even that seems unenforced in these repositories, I reckon).
The attacker went through the hassle to compromise a very widely used package, but use a non standard port (8000) on their C2...
If you plan to do something like that, use 443 at least, many corporate network do not filter this one ;)
NPM should learn from Linux distribution package managers.
Have a branch called testing, and packages stay in testing for few weeks, after which they go to stable. That is how many Linux distributions handle packages. It would have prevented many of these.
Advising every user of npm/pnpm to change their settings and set their own cooldown periods is not a real choice.
Not all distributions work with a staging repository, and it's not really intended for this purpose either.
Besides there's always a way to immediately push a new version to stable repositories. You have to in order to deal with regressions and security fixes.
I know not all, but Debian/Ubuntu/Fedora does, and while the intended purpose of multi-stage releases is not necessarily security but stability, it still does help up with security too. Because third parties can look and scan the dependencies while they are still not in stable.
Most of the supply chain vulnerabilities that ended up in the NPM would have been mitigated with having mandatory testing / stable branches, of course there needs to be some sort of way to skip the testing but that would be rather rare and cumbersome and audited, like it is in Linux distributions too.
NPM is one big AUR, where anyone can submit arbitrary unverified code. The difference is that AUR is intentionally harder to use to prevent catastrophic one-line installs.
Is a "AUR" now just how we name unaudited software repositories?
Just to note, if we're talking about Linux Distributions. There's also COPR in Fedora, OBS for OpenSUSE (and a bunch of other stuff, OBS is awesome), Ubuntu has PPAs. And I am sure there's many more similar solutions.
I am not saying this is the reason for this compromise but the sudden explosion of coding assistant like claude code, and tools like openclaw is teaching entire crop of developers (and users) that it is ok to have sensitive credentials .env files.
Axios has a long history, and is included in a lot of code, also in indirect dependencies. Just check its npm page: it has 174025 dependents as of this moment, including a lot of new packages (I see openclaw and mcp related packages in the list).
And with LLMs generating more and more code, the risk of copying old setups increases.
Can we get a non-AI-generated article for this? I think the aikido one might be fine, but if there’s a more official source let’s use that in lieu of this AI nonsense.
To have an initial smoke test, why not run a diff between version upgrades, and potentially let an llm summarise the changes? It’s a baffling practice that a lot of developers are just blindly trusting code repos to keep the security standards. Last time I installed some npm package (in a container) it loaded 521 dependencies and my heart rate jumped a bit
Is this the first time you have ever thought about the idea of supply chain attacks? This is the first thought 90% of people have and it doesn't work. Too much work to manually verify diffs and LLMs aren't good enough at this yet.
In light of these nonstop supply chain attacks:
Tonight I created /supply-chain-audit
-- A simple claude code skill that fetches info on the latest major package vulnerability, then scans your entire ~/ and gives you a report on all your projects.
Just sanity checking - if I only ever install axios in a container that has no secrets mounted in to its env, is there any real way I can get pwned by this kind of thing?
I am now migrating all my unencrypted secrets on my machines to encrypted ones. If a tool supports scripted credential providers (e.g. aws-cli or Ansible), I use that feature. Otherwise, I wrap the executable with a script that runs gpg --decrypt and injects an environment variable.
That way, I can at least limit the blast radius when (not if) I catch an infostealer.
I believe compartmentalized operating systems like Qubes are the future for defending against these kinds of attacks.
Storing your sensitive data on a single bare-metal OS that constantly downloads and runs packages from unknown maintainers is like handing your house key out to a million people and hoping none of them misuse it.
> Both versions were published using the compromised npm credentials of a lead axios maintainer, bypassing the project's normal GitHub Actions CI/CD pipeline.
Doesn’t npm mandate 2FA as of some time last year? How was that bypassed?
I wonder if this has any connection with the recent string of attacks including the FBI director getting hacked. The attack surface is large, executed extremely cleanly - almost as if done by a high profile state sponsored actor, just like in Hollywood movies.
> This creates a secondary deception layer. After infection, running npm list in the project directory will report plain-crypto-js@4.2.0 — because npm list reads the version field from the installed package.json, which now says 4.2.0. An incident responder checking installed packages would see a version number that does not match the malicious 4.2.1 version they were told to look for, potentially leading them to conclude the system was not compromised.
WTF!!!! gaslighting your victims into believing they are not victims. the ingenuity of this is truly mindblowing. I am shocked at such thing is even allowed. like packages should not be able to modify their contents while they are being instaleld.
Guix saves you from this. You can import NPM packages in a container (not even touching $HOME) and giving you a shell on the spot with just the dependencies and nothing more.
Learn about 'guix import'.
Oh, and you can install Guix on any GNU/Linux distro.
Just a reminder that you can run most node things with deno run and have opt in permissions, audit trail and even external permission system integration now. The gotcha is that "deno task <<some package.json script>>" will NOT execute with this model which I find extremely unintuitive and had me thinking deno abandoned its sandbox for nodejs compatibility completely.
This is why Node.js is completely unsuitable as backend.
Until recently, there wasn’t even a standard Promise-based HTTP client. Why should we need to download a library just to make a simple HTTP request? It’s because Node.js’s standard library is too limited, leading to an explosive growth in third-party libraries. As a result, it’s vulnerable to security attacks, and maintaining it in an enterprise environment becomes a major challenge.
Let’s use .NET or Go. Why use JavaScript outside of the browser when there are excellent backend environments out there?
Absolutely. If you ever did a npm install on a project using one of the affected axios versions, your entire system may be compromised.
> The malicious versions inject a new dependency, plain-crypto-js@4.2.1, which is never imported anywhere in the axios source code. Its sole purpose is to execute a postinstall script that acts as a cross platform remote access trojan (RAT) dropper, targeting macOS, Windows, and Linux. The dropper contacts a live command and control server and delivers platform specific second stage payloads. After execution, the malware deletes itself and replaces its own package.json with a clean version to evade forensic detection.
The NPM ecosystem is a joke. I don't even want anything to do with it, because my stack is fully Elixir. But, just because of this one dependency that is used in some interfaces within my codebase, I need to go back to all my apps and fix it. Sigh.
JavaScript, its entire ecosystem is just a pack of cards, I swear. What a fucking joke.
But also have a regular review of your dependencies to update them when necessary, because as bad as compromised packages may be things do have vulnerabilities occasionally, and upgrading things that are a long way out-of-date can be quite hard.
I lost respect for Axios when they made a breaking change in a patch release. Digging into the root cause, I found the maintainer had approved an outside PR with an obvious AI slop PR description: https://github.com/axios/axios/issues/7059
Looks like the maintainer wasn't just careless when reviewing PRs.
I have a few projects which rely on npm (and react) and every few months I have to revisit them to do an update and make sure they still build, and I am basically done with npm and the entire ecosystem at this point.
Sure, its convenient to have so much code to use for basic functionality - but the technical debt of having to maintain these projects is just too damn high.
At this point I think that, if I am forced to use javascript or node for a project, I reconsider involvement in that project. Its ecosystem is just so bonkers I can't justify the effort much longer.
There has to be some kind of "code-review-as-a-service" that can be turned on here to catch these things. Its just so unproductive, every single time.
It’s only a losing strategy if you assume everyone universally adopts the slow strategy, and no research teams spot it in the interim. For things with large splash radius, that’s unrealistic, so defenders have an information advantage.
Makes actual security patches tougher to roll out though - you need to be vigilant to bypass the slowdown when you’re actually fixing a critical flaw. But nobody said this would be easy!
Has anyone tested general purpose malware detection on supply chains ? Like clamscan . I tried to test the LiteLLM hack but the affected packages had been pulled. Windows Defender AV has an inference based detector that may work when signatures have not yet been published
I second this question. I usually scan our containers with snyk and guarddog, and have wondered about guarddog in particular because it adds so much build time.
Hopefully desktop Linux users will start to understand that malware actually does exist for Linux and that their operating system is doing nothing to protect them from getting RATed.
Yet npm isn't using them allowing this RAT to work. It is not secure by default. It requires every app to manually opt in to being secure. This opt in approach to security puts desktop Linux decades behind in regards to security. Not ahead.
Take for example iOS and Android. All apps are sandboxed by default. You can't make a program that just steals all of your credentials like you can on desktop Linux. Having security tools means nothing if they aren't being used.
Skipping Node sounds nice. PyPI and RubyGems have had the same mess, and npm gets more headlines because it is huge and churns fast, so you see more fresh landmines and more people stepping on them. Unless you plan to audit every dep and pin versions yourself, you're mostly trading one supply chain mess for another, with a tiny bit of luck and a differnt logo.
Log4Shell was hardly a supply-chain attack - just a latent bug in a widely-used library. That can happen anywhere.
Maven to this day represents my ideal of package distribution. Immutable versions save so much trouble and I really don't understand why, in the age of left-pad, other people looked at that and said, "nah, I'm good with this."
Completely agree. NPM has the only registry where massive supply chain attacks happen several times a year. Mainly the fault lies with NPM itself, but much of it is just a terrible opsec culture in the community.
Most package.jsons I see have semver operators on every dependency, so patches spread incredibly quickly. Package namespacing is not enforced, so there is no way of knowing who the maintainer is without looking it up on the registry first; for this reason many of the most popular packages are basically side projects maintained by a single developer*. Post-install scripts are enabled by default unless you use pnpm or bun.
When you combine all these factors, you get the absolute disaster of an ecosystem that NPM is.
*Not really the case for Axios as they are at least somewhat organized and financed via sponsors.
The semantics are irrelevant. The effect is what's important: Hijacking widely used software to exploit systems. The OC is somehow under the illusion that avoiding JS altogether is a silver bullet for avoiding this.
Other languages have package managers (perl) and there are package managers in existence that are not so vulnerable to this issue. IMO, it stems from one place: Transitive dependencies and general opaqueness of the issue.
In package managers like pacman, apt, apk,... it's easier to catch such issue. They do have postinstall scripts, but it's part of the submission to the repo, not part of the project. Whatever comes from the project is hashed, and that hash is also visible as part of the submission. That makes it a bit difficult to sneak something. You don't push a change, they pull yours.
The issues have everything to do with npm as a platform and nothing with JS as a language. You can use JS without npm. Saying you'll escape supply chain attacks by not using JS is like saying you'll be saved from an car crash with a parachute.
local [fuction][Password and Key and DMS]
Axes [Password and K
[UserID] --1234567890-- [Hacking error Message -- Hello --
hacker typer --97283710-- Security
PSA: npm/bun/pnpm/uv now all support setting a minimum release age for packages.
I also have `ignore-scripts=true` in my ~/.npmrc. Based on the analysis, that alone would have mitigated the vulnerability. bun and pnpm do not execute lifecycle scripts by default.
Here's how to set global configs to set min release age to 7 days:
(Side note, it's wild that npm, bun, and pnpm have all decided to use different time units for this configuration.)If you're developing with LLM agents, you should also update your AGENTS.md/CLAUDE.md file with some guidance on how to handle failures stemming from this config as they will cause the agent to unproductively spin its wheels.
> (Side note, it's wild that npm, bun, and pnpm have all decided to use different time units for this configuration.)
First day with javascript?
You mean first 86,400 seconds?
You have to admire the person who designed the flexibility to have 87239 seconds not be old enough, but 87240 to be fine.
I actually think it is not too bad a design, because seconds are the SI base unit for time. Putting something like "x days" requires additional parsing steps and therefore complexity in the implementation. Either knowing or calculating how many seconds there are in a day can be expected of anyone touching a project or configuration at this level of detail.
Seconds are also unambiguous. Depending on your chosen definition, "X days" may or may not be influenced by leap seconds and DST changes.
I doubt anyone cares about an hour more or less in this context. But if you want multiple implementations to agree talking about seconds on a monotonic timer is a lot simpler
I came here to argue the opposite. Expressing it in seconds takes away questions about time zones and DST.
I think you're incorrect to say that second are also ambiguous. Maybe what you mean is that days are more practical, but that seems very much a personal preference.
I think you misread the comment you're replying to.
> seconds are the SI base unit for time
True. But seconds are not the base unit for package compromises coming to light. The appropriate unit for that is almost certainly days.
that kind of complexity is always worth it. Every single time. It's user time that you're saving and it also makes config clearer for readers and cuts out on "too many/little zeroes on accident" errors
It's just library for handling time that 98% of the time your app will be using for something else.
This is the difference between thinking about the user experience and thinking just about the technical aspect
OP should be glad a new time unit wasn't invented
Workdays! Think about it, if you set the delay in regular days/seconds the updated dependency can get pulled in on a weekend with only someone maybe on-call.
(Hope your timezones and tzdata correctly identifies Easter bank holiday as non-workdays)
> Workdays!
This is javascript, not Java.
In JavaScript something entirely new would be invented, to solve a problem that has long been solved and is documented in 20+ year old books on common design patterns. So we can all copy-paste `{ or: [{ days: 42, months: 2, hours: "DEFAULT", minutes: "IGNORE", seconds: null, timezone: "defer-by-ip" }, { timestamp: 17749453211*1000, unit: "ms"}]` without any clue as to what we are defining.
In Java, a 6000LoC+ ecosystem of classes, abstractions, dependency-injectables and probably a new DSL would be invented so we can all say "over 4 Malaysian workdays"
In before someone thinks it's a joke, the most commonly used logging library in Java had LDAP support in format scripts enabled by default" (which resulted, of course in CVE)
But you know that Java solution will continue working even after we no longer use the Gregorian Calendar, the collapse and annexation of Malaysia to some foreign power, and then us finally switching to a 4-day work week; so it'd be worth it.
It probably won’t work correctly from the get go. But it can be debugged everywhere so that’s good.
... and since it was architectured to allow runtime injection-patching of events before they hit the enterprise-service-bus, everyone using this library must first set fourteen ENV vars in their profile, and provide a /etc/java/springtime/enterprise-workday-handling/parse-event-mismatch.jar.patch. Which should fix the bug for you.
You can find the patch files for your OSs by registering at Oracle with a J3EE8.4-PatchLibID (note, the older J3EE16-PatchLib-ids aren't compatible), attainable from your regional Oracle account-manager.
This isn’t even remotely funny.
JavaScript Temporal. Not sure knowing what a "workday" is in each timezone is in it's scope but it's the much needed and improved JS, date API (granted with limited support to date)
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Why would it get pulled in over the weekend? What automatic deployments are you running if there also isn't a human working to get it out?
Do you run automatic dependency updates over the weekend? Wouldn't you rather do that during fully-staffed hours?
And we also need localization. Each country can have their own holidays
And we need groups of locales for teams that are split across multiple locations; e.g.:
Hopefully "es" will have Siesta support too.
Might be better to calculate them separately for each locale and then tie-break with your own approach (min/max/avg/median/etc.)
Don't forget about regional holidays, which might follow arbitrary borders that don't match any of the official subdivisions of the country. Or may even depend on the chosen faith of the worker
Pulaski day in Illinois. Or Reds Opening Day in Cincinnati.
If we're taking suggestions, I'd like to propose "parsec" (not to be confused with the unit of distance of the same name)
That way Han Solo can make sense in the infamous quote.
EDIT: even Gemini gets this wrong:
> In Star Wars, a parsec is a unit of distance, not time, representing approximately 3.26 light-years
It was already fine, because it’s a metric defined on a submanifold of relativistic spacetime.
> That way Han Solo can make sense in the infamous quote.
They explained it in the Solo movie.
https://www.reddit.com/r/MovieDetails/comments/ah3ptm/solo_a...
That had more or less been the explanation in the books for decades, and even in George Lucas' notes from 1977:
> It's a very simple ship, very economical ship, although the modifications he made to it are rather extensive – mostly to the navigation system to get through hyperspace in the shortest possible distance (parsecs).
Making a whole movie just to retcon the parsec misuse in Ep IV was a choice
They made a movie to make money. I doubt anyone holding the purse strings cared one iota if that bit were corrected or not. It’s not really a retcon either because they didn’t change anything.
N multiplications of dozen-second
To me it sounds safer to have different big infra providers with different delays, otherwise you still hit everyone at the same time when something does inevitably go undetected.
And the chances of staying undetected are higher if nobody is installing until the delay time ellapses.
It's the same as not scheduling all cronjobs to midnight.
Pnpm did this first but I’m glad to see all the others follow suit
For anyone wondering, you need to be on npm >= 11.10.0 in order to use it. It just became available Feb 11 2026
https://github.com/npm/cli/releases/tag/v11.10.0
About the use of different units: next time you choose a property name in a config file, include the unit in the name. So not “timeout” but “timeoutMinutes”.
Or require the value to specify a unit.
At that point, you're making all your configuration fields strings and adding another parsing step after the json/toml/yaml parser is done with it. That's not ideal either; either you write a bunch of parsing code (not terribly difficult but not something I wanna do when I can just not), or you use some time library to parse a duration string, in which case the programming language and time library you happen to use suddenly becomes part of your config file specification and you have to exactly re-implement your old time handling library's duration parser if you ever want to switch to a new one or re-implement the tool in another language.
I don't think there are great solutions here. Arguably, units should be supported by the config file format, but existing config file formats don't do that.
TOML has a datetime type (both with or without tz), as well as plain date and plain time:
We should extend it with durations: And like for datetimes, we should have a readable variant: Edit: discussed in detail here: https://github.com/toml-lang/toml/issues/514> adding another parsing step after the json/toml/yaml parser is done with it. That's not ideal either
I'd argue that it is ideal, in the sense that it's the sweet spot for a general config file format to limit itself to simple, widely reusable building blocks. Supporting more advanced types can get in the way of this.
Programs need their own validation and/or parsing anyway, since correctness depends on program-specific semantics and usually only a subset of the values of a more simply expressed type is valid. That same logic applies across inputs: config may come from files, CLI args, legacy formats, or databases, often in different shapes. A single normalization and validation path simplifies this.
General formats must also work across many languages with different type systems. More complex types introduce more possible representations and therefore trade-offs. Even if a file parser implements them correctly (and consistently with other such parsers), it must choose an internal form that may not match what a program needs, forcing extra, less standard transformation and adding complexity on both sides for little gain.
Because acceptable values are defined by the program, not the file, a general format cannot fully specify them and shouldn’t try. Its role is to be a medium and provide simple, human-usable (for textual formats), widely supported types, avoid forcing unnecessary choices, and get out of the way.
All in all, I think it can be more appropriate for a program to pick a parsing library for a more complex type, than to add one consistently to all parsers of a given file format.
Another parsing step is the common case. Few parameters represent untyped strings where all characters and values are valid. For numbers as well, you often have a limited admissible range that you have to validate for. In the present case, you wouldn’t allow negative numbers, and maybe wouldn’t allow fractional numbers. Checking for a valid number isn’t inherently different from checking for a regex match. A number plus unit suffix is a straightforward regex.
timeoutMs is shorter ;)
You guys can't appreciate a bad joke
Megaseconds are about the right timescale anyway
What megaseconds? They clearly meant the Microsoft-defined timeout.
timoutμs is even better. People will learn how to type great symbols.
They wouldn't have to, if the file format accepted floats in proper exponential format.
Yes timout indeed!
not timeout at all is even shorter.
Is there a way to do that per repo for these tools ? We all know how user sided configuration works for users (they usually clean it whenever it goes against what they want to do instead of wondering why it blocks their changes :))
At least with npm, you can have a .npmrc per-repo
It's wild that none of these are set by default.
I know 90% of people I've worked with will never know these options exist.
That would likely mean same amount of people get the vulnerability, just 7 days later.
If everyone or a majority of people sets these options, then I think issues will simply be discovered later. So if other people run into them first, better for us, because then the issues have a chance of being fixed once our acceptable package/version age is reached.
min release age to 7 days about patch releases exposes you to the other side of the coin, you have an open 7 days window on zero-day exploits that might be fixed in a security release
At least with pnpm, you can specify minimumReleaseAgeExclude, temporarily until the time passes. I imagine the other package managers have similar options.
[1]: https://pnpm.io/settings#minimumreleaseageexclude
I haven't checked, but it would be surprising that the min-release-age applies to npm audit and equivalent commands
Exactly what I thought too when I read this...
Urgent fix, patch released, invisible to dev team cause they put in a 7 day wait. Now our app is vulnerable for up to 7 days longer than needed (assuming daily deploys. If less often, pad accordingly). Not a great excuse as to why the company shipped an "updated" version of the app with a standing CVE in it. "Sorry we were blinded to the critical fix because set an arbitrary local setting to ignore updates until they are 7 days old". I wouldn't fire people over that, but we'd definitely be doing some internal training.
Not really an issue though right because virtually none of these have lasted more than 1-2 days before being discovered?
Out of the frying pan and into the frier.....
and for yarn berry
If everyone avoids using packages released within the last 7 days, malicious code is more likely to remain dormant for 7 days.
What do you base that on? Threat researchers (and their automated agents) will still keep analyzing new releases as soon as they’re published.
Their analysis was triggered by open source projects upgrading en-masse and revealing a new anomalous endpoint, so, it does require some pioneers to take the arrows. They didn't spot the problem entirely via static analysis, although with hindsight they could have done (missing GitHub attestation).
A security company could set up a honeypot machine that installs new releases of everything automatically and have a separate machine scan its network traffic for suspicious outbound connections.
The fact threat researchers and especially their automated agents are not all that good at their jobs
Those threat researchers and their autonomous agents caught this axios release.
> What do you base that on?
The entire history of malware lol
that's why people are telling others to use 7 days but using 8 days themselves :)
brb, switching everything to 9 days
That is 3D chess level type shit. xD
Genius
Worth noting this attack was caught because people noticed anomalous network traffic to a new endpoint. The 7-day delay doesn't just give scanners time, it gives the community time to notice weird behavior from early adopters who didn't have the delay set.
It's herd immunity, not personal protection. You benefit from the people who DO install immediately and raise the alarm
But wouldn't the type of people that notifes anomalous network activity be exactly the type of people who add a 7 day delay because they're security conscious?
I suspect most packages will keep a mix of people at 7 days and those with no limit. That being said, adding jitter by default would be good to these features.
>adding jitter by default would be good
This became evident, what, perhaps a few years ago? Probably since childhood for some users here but just wondering what the holdup is. Lots of bad press could be avoided, or at least a little.
They’re usually picked up by scanners by then.
Most people won’t.
7 days gives ample time for security scanning, too.
This highly depends on the detection mechanism.
> If everyone avoids using packages released within the last 7 days
Which will never even come close to happening, unless npm decides to make it the default, which they won't.
lol with mise I used a fourth time unit: https://mise.jdx.dev/configuration/settings.html#install_bef...
I think the npm doesn't support end of line comments, so
actually doesn't set it at all, please edit your comment.EDIT: Actually maybe it does? But it's weird because
`npm config list -l` shows: `min-release-age = null` with, and without the comment. so who knows ¯\_(ツ)_/¯
ok, it works, only the list function shows it as null...
Where in the pnpm documentation does it say that it ignores scripts by default?
From https://pnpm.io/cli/install#--ignore-scripts:
> Default: *false*
Run npm/pnpm/bun/uv inside a sandbox.
There is no reason to let random packages have full access to your machine
Sandboxing by to default world be really nice. One of the things I really appreciate about Claude Code is its permissions model
Props to uv for actually using the correct config path jfc what is “bunfig”
Silly portmanteau of "bun" and "config"
The config for uv won't work. uv only supports a full timestamp for this config, and no rolling window day option afaik. Am I crazy or is this llm slop?
https://docs.astral.sh/uv/concepts/resolution/#dependency-co...
> Define a dependency cooldown by specifying a duration instead of an absolute value. Either a "friendly" duration (e.g., 24 hours, 1 week, 30 days) or an ISO 8601 duration (e.g., PT24H, P7D, P30D) can be used.
My bad. This works for per project configuration, but not for global user configuration.
It should work for global configuration too, please file an issue if you’re observing otherwise.
(Make sure you’re on a version that actually supports relative times, please!)
This is what tripped me up. I added that config and then got this error:
error: Failed to parse: `.config/uv/uv.toml` Caused by: TOML parse error at line 1, column 17 | 1 | exclude-newer = "7 days" | ^^^^^^^^ failed to parse year in date "7 days": failed to parse "7 da" as year (a four digit integer): invalid digit, expected 0-9 but got
I was on version 0.7.20, so I removed that line, ran "uv self update" and upgraded to 0.11.2 and then re-added the config and it works fine now.
Yeah, that error message isn’t ideal on older versions, but unfortunately there’s no way to really address that. But I’m glad it’s working for you on newer versions.
I think it should work at the user config level too:
> If project-, user-, and system-level configuration files are found, the settings will be merged, with project-level configuration taking precedence over the user-level configuration, and user-level configuration taking precedence over the system-level configuration.
https://docs.astral.sh/uv/concepts/configuration-files/
npm is claiming this doesn’t exist
Make sure you're on version 11.10 or later?
Good luck with any `npm audit` in a pipeline. Sometimes you have to pull the latest release because the previous one had a critical vulnerability.
"Batteries included" ecosystems are the only persistent solution to the package manager problem.
If your first party tooling contains all the functionality you typically need, it's possible you can be productive with zero 3rd party dependencies. In practice you will tend to have a few, but you won't be vendoring out critical things like HTTP, TCP, JSON, string sanitation, cryptography. These are beacons for attackers. Everything depends on this stuff so the motivation for attacking these common surfaces is high.
I can literally count on one hand the number of 3rd party dependencies I've used in the last year. Dapper is the only regular thing I can come up with. Sometimes ScottPlot. Both of my SQL providers (MSSQL and SQLite) are first party as well. This is a major reason why they're the only sql providers I use.
Maybe I am just so traumatized from compliance and auditing in regulated software business, but this feels like a happier way to build software too. My tools tend to stay right where I left them the previous day. I don't have to worry about my hammer or screw drivers stealing all my bitcoin in the middle of the night.
Irony is that Node has no need for Axios, native fetch support has been there for years, so in terms of network requests it is batteries included.
People use axios or ky because with fetch you inevitably end up writing a small wrapper on top of it anyway.
Some might say the tradeoff of writing a small wrapper is worth it given what’s been demonstrated here.
> In practice you will tend to have a few, but you won't be vendoring out critical things like HTTP, TCP, JSON, string sanitation, cryptography
Unless you are Python, where the standard library includes multiple HTTP libraries and everyone installs the requests package anyways.
Few languages have good models for evolving their standard library, so you end up with lots of bad designs sticking around forever. Libraries are much easier to evolve, giving them the advantage in terms of developer UX and performance.
What type of developer chooses UX and performance over security? So reckless.
I removed the locks from all the doors, now entering/exiting is 87% faster! After removing all the safety equipment, our vehicles have significantly improved in mileage, acceleration and top speed!
>What type of developer chooses UX and performance over security? So reckless.
Initially I assumed this is sarcastic, but apparently not. UX and performance is what programmers are paid to do! Making sure UX is good is one of the most important things in programmer job.
While security is a moving target, a goal, something that can never be perfect, just "good enough" (if NSA wants to hack you, they will). You make it sound like installing third party packages is basically equivalent to a security hole, while in practice the risk is low, especially if you don't overdo it.
Wild to read extreme security views like that, while at the same time there are people here that run unconstrained AI agents with --dangerous-skip-confirm flags and see nothing wrong with it.
Better developer UX can directly lead to better safety. "You are holding it wrong" is a frequent source of security bugs, and better UX reduces the ways you can hold it wrong, or at least makes you more likely to hold it the right way
I'm pretty sure it's really one HTTP library: urllib.request is built on top of http.client. But the very Java-inspired API for the former is awful.
The other thing that keeps coming up is the github-code-is-fine-but-the-release-artifact-is-a-trojan issue. It really makes me question if "packages" should even exist in JavaScript, or if we could just be importing standard plain source code from a git repo.
I understand why this doesn't work well with legacy projects, but it's something that the language could strive towards.
> I understand why this doesn't work well with legacy projects, but it's something that the language could strive towards.
Why wouldn't that work well with legacy projects? In fact, the projects I was a part of that I'd call legacy nowadays, was in fact built by copy-and-pasting .js libraries into a "vendor/" directory, and that's how we shipped it as well, this was in the days before Bower (which was the npm of frontend development back in the day), vendoring JS libs was standard practice, before package managers became used in frontend development too.
Not sure why it wouldn't work, JavaScript is a very moldable language, you can make most things work one way or another :)(
This is a rather superlative and tunnel vision, "everything is a nail because I'm a hammer" approach. The truth is this is an exceedingly difficult problem nobody has adequately solved yet.
I think the AI tooling is, if not completely solving sandboxing, at least making the default much better by asking you every time they want to do something and providing files to auto-approve certain actions.
Package managers should do the same thing
Fully agree with this! I think today .NET is probably the most batteries included platform you can get. This means that even if you use third-party libraries, these typically depend only on first-party dependencies, making it much less likely for something shady to sneak in.
With the notable exception of cross-platform audio.
To me, I really like Golang's batteries included platform. I am not sure about .NET though
C#'s LINQ (code as data, like LISP) wins over golang for any type of data access. Strongly-typed, language-native queries. Go has its own advantages though.
And now with NativeAOT, you can use C# like go - you don't need to ship the CLR.
What are some examples of batteries-included languages that folk around here really feel productive in and/or love? What makes them so great, in your opinion?
(Leaving aside thoughts on language syntax, compile times, tooling etc - just interested in people's experiences with / thoughts on healthy stdlibs)
Go is well known for its large and high quality std lib
These are the big ones I use, specifically because of the standard libraries:
Python (decent standard library) - It's pretty much everywhere. There's so many hidden gems in that standard library (difflib, argparse, shlex, subprocess, cmd)
C#/F# (.NET)
C# feels so productive because of how much is available in .NET Core, and F# gets to tag along and get it all for free too. With C# you can compile executables down to bundle the runtime and strip it down so your executables are in the 15 MiB range. If you have dotnet installed, you can run F# as scripts.
These are definitely some good thoughts, thanks!
Do you worry at all about the future of F#? I've been told it's feeling more and more like a second-class citizen on .NET, but I don't have much personal experience.
I agree. Got downvoted a lot the other day for proposing Node should solve fundamental needs.
But javascript is batteries included in this case, you can use xmlhttprequest or fetch
> "Batteries included" ecosystems are the only persistent solution to the package manager problem.
The irony in this case is that axios is not really needed now given that fetch is part of the JS std lib.
For a lot of code, I switched to generating code rather than using 3rd party libraries. Things like PEG parsers, path finding algorithms, string sanitizers, data type conversion, etc are very conveniently generated by LLMs. It's fast, reduces dependencies, and feels safer to me.
Or find the best third party library and copy the code from a widely used version that has been out long enough to have been well tested into your source tree.
The problem is not third party libraries. It is updating third party libraries when the version you have still works fine for your needs.
Ah, so you've traded the possibility of bad dependencies for certainty.
How can you come to that conclusion, given the specific examples I have given, which are tedious to write, but easy to proof-read and test?
Things like unsafe string handling and data type conversion are some of the richest veins of security vulnerabilities I could possibly think of, and you're letting an LLM replace battle-handed library code because it "feels safer".
I admire your confidence in your ability to proof-read reams of "tedious" code, and come up with comprehensive tests for obscure edge cases. The kind of edge cases solved in established libraries for decades, which is why the average coder no longer even thinks about them. The kinds of edge cases that pay a black hat's bills.
The only reason you feel safer is because `pnpm audit` keeps you up to date on CVEs that affect you, which is unsettling, whereas your LLM whispers in your ear: "You are absolutely correct. This is production-ready code. It's not just secure — it's the right solution for your business. It's correct, safe, and performant. Don't overthink it — ship it. *green tick emoji*"
I'm not against LLMs, but there are good uses for LLMs, and then there's horror stories like this.
Remember, our objective function here is “feels safe.”
Honestly, you can get pretty far with just Bun and a very small number of dependencies. It’s what I love most about Bun. But, I do agree with you generally. .NET is about as good as I’ve ever seen for being batteries included. I just hate the enterprisey culture that always seems to pervade .NET shops.
I agree about the culture. If I take my eye off the dev team for too long, I'll come back and we'll be using entity framework and a 20 page document about configuring code cleanup rules in visual studio.
Entity framework is pretty good.
What kind of apps do you build / industry etc?
There’s a recurrent pattern with these package compromises: the attacker exfiltrates credentials during an initial phase, then pivots to the next round of packages using those credentials. That’s how we saw them make the Trivy to LiteLLM leap (with a 5 day gap), and it’ll almost certainly be similar in this case.
The solution to this is twofold, and is already implemented in the primary ecosystems being targeted (Python and JS): packagers should use Trusted Publishing to eliminate the need for long lived release credentials, and downstreams should use cooldowns to give security researchers time to identify and quarantine attacks.
(Security is a moving target, and neither of these techniques is going to work indefinitely without new techniques added to the mix. But they would be effective against the current problems we’re seeing.)
In this case, the author's NPM account was taken over, email address changed to one the attacker controls, and the package was manually published.
Since the attacker had full control of the NPM account, it is game over - the attacker can login to NPM and could, if they wanted, configure Trusted Publishing on any repo they control.
Axios IS using trusted publishing, but that didn't do anything to prevent the attack since the entire NPM account was taken over and config can be modified to allow publishing using a token.
There are solutions, the problem is almost always discipline.
I don’t know what this means. Discipline is good, but I think you need to have good tools/primitives in place to help people exercise discipline.
(The classic example being passwords: we wouldn’t need MFA is everybody just “got good” and used strong/unique passwords everywhere. But that’s manifestly unrealistic, so instead we use our discipline budget on getting people to use password managers and phishing-resistant MFA.)
I can't even imagine the scale of the impact with Axios being compromised, nearly every other project uses it for some reason instead of fetch (I never understood why).
Also from the report:
> Neither malicious version contains a single line of malicious code inside axios itself. Instead, both inject a fake dependency, plain-crypto-js@4.2.1, a package that is never imported anywhere in the axios source, whose only purpose is to run a postinstall script that deploys a cross-platform remote access trojan (RAT)
Good news for pnpm/bun users who have to manually approve postinstall scripts.
> nearly every other project uses it for some reason instead of fetch (I never understood why).
Fetch wasn't added to Node.js as a core package until version 18, and wasn't considered stable until version 21. Axios has been around much longer and was made part of popular frameworks and tutorials, which helps continue to propagate it's usage.
Also it has interceptors, which allow you to build easily reusable pieces of code - loggers, oauth, retriers, execution time trackers etc.
These are so much better than the interface fetch offers you, unfortunately.
You can do all of that in fetch really easily with the init object.
})There are pretty much two usage patterns that come up all the time:
1- automatically add bearer tokens to requests rather than manually specifying them every single time
2- automatically dispatch some event or function when a 401 response is returned to clear the stale user session and return them to a login page.
There's no reason to repeat this logic in every single place you make an API call.
Likewise, every response I get is JSON. There's no reason to manually unwrap the response into JSON every time.
Finally, there's some nice mocking utilities for axios for unit testing different responses and error codes.
You're either going to copy/paste code everywhere, or you will write your own helper functions and never touch fetch directly. Axios... just works. No need to reinvent anything, and there's a ton of other handy features the GP mentioned as well you may or may not find yourself needing.
Interceptors are just wrappers in disguise.
Convenience is a thing, but it doesn't require a massive library.but it does for massive DDoS :p
That fetch requires so many users to rewrite the same code - that was already handled well by every existing node HTTP client- says something about the standards process.
It could also be trivially written for XMLHttpRequest or any node client if needed. Would be nice if they had always been the same, but oh well - having a server and client version isn't that bad.
Because it is so few lines it is much more sensible to have everyone duplicate that little snippet manually than import a library and write interceptors for that...
(Not only because the integration with the library would likely be more lines of code, but also because a library is a significantly liability on several levels that must be justified by significant, not minor, recurring savings.)
that's such a weak argument. you can write about 20 lines of code to do exactly this without requiring a third party library.
> Likewise, every response I get is JSON.
fetch responses have a .json() method. It's literally the first example in MDN: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/U...
It's literally easier than not using JSON because I have to think about if I want `repsponse.text()` or `response.body()`.
Helper functions seem trivial and not like you’re reimplementing much.
Don't be silly, this is the JS ecosystem. Why use your brain for a minute and come up with a 50 byte helper function, if you can instead import a library with 3912726 dependencies and let the compiler spend 90 seconds on every build to tree shake 3912723 out again and give you a highly optimized bundle that's only 3 megabytes small?
> usage patterns
IMO interceptors are bad. they hide what might get transformed with the API call at the place it is being used.
> Likewise, every response I get is JSON. There's no reason to manually unwrap the response into JSON every time.
This is not true unless you are not interfacing with your own backends. even then why not just make a helper that unwraps as json by default but can be passed an arg to parse as something else
One more use case for Axios is it automatically follows redirects, forwarding headers, and more importantly, omiting or rewriting the headers that shouldn't be forwarded for security reasons.
What does an interceptor in the RequestInit look like?
It also supports proxies which is important to some corporate back-end scenarios
fetch supports proxies
> I can't even imagine the scale of the impact with Axios being compromised, nearly every other project uses it for some reason instead of fetch (I never understood why).
You can remember this answer for every time you ask same question again:
"Coz whatever else/builtin was before was annoying enough for common use cases"
>(I never understood why).
Because axios existed before the builtin fetch, and so there's a lot of stackoverflow answers explaining how to use fetch, and the llm models are trained on that, so they will write axios requests instead of fetch
> Good news for pnpm/bun users who have to manually approve postinstall scripts.
Would they not have approved it for earlier versions? But also wouldn't the chance of addition automatic approval be high (for such a widely used project)?
The prompt would be to approve the new malicious package (plain-crypto-js)'s scripts, too, which could tip users off that something was fishy. If they were used to approving one for axios and the attackers had just overwrote axios's own instead of making a new package, it would probably catch people out.
Assuming axios didn't have a postinstall script before, it wouldn't have been approved for a previous version. If you ignore it, you ignore it, but postinstall scripts are relatively rare in npm deps, so it would seem a bit out of place when the warning pops up.
Can't speak for other devs but I like to read postinstall scripts or at least put them through an LLM if they're too hard to grok.
It's also a little context dependent, for example if I was using Axios and I see a prompt to run the plain-crypto-js postinstall script, alarm bells would instantly ring, which would at least make me look up the changelog to see why this is happening.
In most cases I don't even let them run unless something breaks/doesn't work as expected.
Does pnpm block postinstall on transitive deps too or just top-level? We have it configured at work but I've never actually tested whether it catches scripts from packages that get pulled in as sub-dependencies.
It prompts for transitive dependencies, too. I have never had workerd as a direct dependency of any project of mine but I get prompted to approve its postinstall script whenever I install cloudflare's wrangler package (since workerd needs to download the appropriate Workers runtime for your platform).
From what I can tell, it blocks it everywhere.
That's solid, really helps lock down the supply chain attack surface. Do you ever end up having to whitelist anything that legitimately needs to run on install?
After using pnpm for years (at least 5, don't remember exactly), I've only ever had to whitelist one library that uses a postinstall script to download a native executable for your system. And even this is not necessary, it's just poorly designed.
For example, esbuild and typescript 7 split binaries for different systems and architectures into separate packages, and rely on your package manager to pull the correct one.
Setting min-release age to 7 days is great, but the only true way to protect from supply chain attacks is restricting network access.
This needs to be done (as we've seen from these recent attacks) in your devenv, ci/cd and prod environments. Not one, or two, but all of these environments.
The easiest way is via using something like kubernetes network policies + a squid proxy to allow limited trusted domains through, and those domains must not be publicly controllable by attackers. ie. github.com is not safe to allow, but raw.githubusercontent.com would be as it doesn't allow data to be submitted to it.
This stops both DNS exfil and HTTP exfil. For your devenv, software like Little Snitch may protect your from these (I'm not 100% on DNS exfil here though). Otherwise run your devenv (ie vscode) as a web server, or containerised + vnc, a VM, etc, with the same restrictions.
Not to beat a dead horse but I see this again and again with dependencies. Each time I get more worried that the same will happen with rust. I understand the fat std library approach won’t work but I really still want a good solution where I can trust packages to be safe and high quality.
If the fat std library is not viable you can only increase security requirements.
Axios has like 100M downloads per week. A couple of people with MFA should have to approve changes before it gets published.
This is the actual answer: stupid cost saving creating an operational risk.
At least then they will have to pay off a dev or something, changes their economic calculus and is additionally illegal
Hosting curated dependencies is a commercially valuable service. Eventually an economy arises where people pay vendors to vet packages.
It's what linux distributions do.
Queue appimage or other packed binary and there go your finetuned packages.
Yes, that why those need to be 100% sandboxed by default (ideally a VM), unless they are provided by distro
what?
Linux distros and BSD ports did that since the 90's. When Linux distros had barely a PM or just tarballs, Infomagic sold 4 CD full of libre software. When I had no internet at home, back in the day I bought 3 DVD's of Debian Sarge for 20 euros, about $20. A bargain, it was the price of a hard-cover best seller book.
GB's of libre software, graphical install, 2.6 kernel, KDE3 desktop, very light on my Athlon 2000 with 256MB of RAM. It was incredible compared to what you got with Windows XP and 120 Euro per seat. Nonfree software and almost empty.
And, well, if for instance I could get read only, ~16TB durable USB drive with tons of Guix packages offline (in a two yearly basis with stable releases) for $200 I would buy them in the spot.
You would say that $200 for a distro it's expensive, but for what it provides, if you are only interested in libre gaming and tools, they amount you save can be huge. I've seen people spend $400 in Steam games because of the Holyday sales...
It already exists; cloudsmith
Why wouldn't the "fat std" thing work? Yes it's hard to design properly, both in scope and actual design (especially for an unstandardized language still moving fast), but throwing the towel and punting the problem to the "free market" of uncurated public repos is even worse.
It's what we call in France "la fête du slip".
PS: that's one reason I try to use git submodules in my Common Lisp projects instead of QuickLisp, because I really see the size of my deptree this way.
Because fat std is rigid, impractical, and annoying.
In practice (e.g. Go) it’s actually pretty good and infinitely preferable to third party everything.
Works just fine in Go.
Fat std library mistakes/warts would likely result in third party packages being used anyway.
Not necessarily, but let's agree that some design faults would happen: you still get the option to use the solid, boring and slightly rusty std instead of another 100 dependencies from the supply chain supermarket.
At work, we're happy with Python's included batteries when we need to make scripts instead of large programs.
So it provides another option, and in worst case it doesn't make situation worse than it is right now?
Yeah, pretty bad idea.
NPM should have a curation mechanism, via staff review or crowdsourcing, where versions of popular packages are promoted to a stable set, like linux distros do. I would only use curated versions if they had such a thing.
An alternative:
- copy the dependencies' tests into your own tests
- copy the code in to your codebase as a library using the same review process you would for code from your own team
- treat updates to the library in the same way you would for updates to your own code
Apparently, this extra work will now not be a problem, because we have AI making us 10x more efficient. To be honest, even without AI, we should've been doing this from the start, even if I understand why we haven't. The excuses are starting to wear thin though.
Just going to put features on hold for a month while I review the latest changes to ffmpeg.
I don't know where you've worked but a hostile and intelligent actor or internal red team would succeed under each of those cases at every job I've worked at.
Defending against a targeted attack is difficult, yes. But these recent campaigns were all directed at everyone. Auditing and inspecting your dependencies does absolutely help thwart that because there will always be people who don't.
Good to know. Where were the places you worked at?
I recommend everyone to use bwrap if you're on linux and alias all package managers / anything that has post build logic with it.
I have bwrap configured to override: npm, pip, cargo, mvn, gradle, everything you can think of and I only give it the access it needs, strip anything that is useless to it anyway, deny dbus, sockets, everything. SSH is forwarded via socket (ssh-add).
This limits the blast radius to your CWD and package manager caches and often won't even work since the malware usually expects some things to be available which are not in a permissionless sandbox.
You can think of it as running a docker container, but without the requirement of having to have an image. It is the same thing flatpak is based on.
As for server deployments, container hardening is your friend. Most supply chain attacks target build scripts so as long as you treat your CI/CD as an untrusted environment you should be good - there's quite a few resources on this so won't go into detail.
Bonus points: use the same sandbox for AI.
Stay safe out there.
This only works for post-install script attacks. When the package is compromised, just running require somewhere in your code will be enough, and that runs with node/java/python and no bwrap.
node is also sandboxed within bwrap I have sandbox -p node if I have to give node access to other folders, I also have sandbox -m to define custom mountpoints if necessary and UNSAFE=1 as a last resort which just runs unsandboxed.
I think firejail is a much more flexible security sandbox than bwrap. It also comes with pre-defined profiles
bwrap is as secure as you want it to be which I think is the primary advantage over anything else.
I wrote a Docker-based sandbox [1] for myself last year to control the blast radius of such malicious packages.
https://github.com/ashishb/amazing-sandbox
Check also https://github.com/wrr/drop which is a higher-level tool than bwrap. It allows you to make such isolated sandboxes with minimal configuration.
This looks nice but I wouldn't trust a very fresh tool to do security correctly.
As a higher-level alternative to bwrap, I sometimes use `flatpak run --filesystem=$PWD --command=bash org.freedesktop.Platform`. This is kind of an abuse of flatpaks but works just fine to make a sandbox. And unlike bwrap, it has sane defaults (no extra permissions, not even network, though it does allow xdg-desktop-portal).
I like the idea of bubblewrap, but my pain point is that it is work to set it up correctly with bind mounts and forwarding necessary environment variables to make the program actually work usefully. Could you share your pip bwrap configuration? It sounds useful.
can't really share a file here, feel free to email me
AFAIK maven doesn’t support post install logic like npm does. You have to explicitly optin with build plugins. It doesn’t let any arbitrary dependency run code on your machine.
some post processors have chains to execution (ex: lombok)
You explicitly opt in by using a compiler plugin. Merely having it as a dependency, like in npm, doesn’t mean it can run code at build time.
> SSH is forwarded via socket
Maybe I misunderstood this point. But the ssh socket also gives access to your private keys, so I see no security gain in that point. Better to have a password protected key.
It's so your private key is not stolen, but you're right passphrase protected keys win anyway. I use hardware keys so this isn't a problem for me to begin with.
Do you have a recommendation for something like bwrap but for macos? I've been trying to use bwrap more on my servers when I remember.
unfortunately not, but there is work being done to support overlays properly I think?
Package managers are a failed experiment.
We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a ton of incredibly useful, non-trivial work for you, while barely increasing your executable's size.
The issue is not dependencies themselves, it's transitive ones. Nobody installs left-pad or is-even-number directly, and "libraries" like these are the vast majority of the attack surface. If you get rid of transitive dependencies, you get rid of the need of a package manager, as installing a package becomes unzipping a few files into a vendor/ folder.
There's so many C libraries like this. Off the top of my head, SQLite, FreeType, OpenSSL, libcurl, libpng/jpeg, stb everything, zlib, lua, SDL, GLFW... I do game development so I'm most familiar with the ones commonly used in game engines, but I'm sure other fields have similarly high quality C libraries.
They also bindings for every language under the sun. Rust libraries are very rarely used outside of Rust, and C#/Java/JS/Python libraries are never used outside their respective language (aside form Java ones in other JVM langs).
They're not a failed experiment. No one has ever "experimented" by making a safe package manager for their new language. And it is not that insane to do so. Very basic things will get you very far:
1. Packages should carry a manifest that declares what they do at build time, just like Chrome extensions do. This manifest would then be used to configure its build environment.
2. Publishers to official registries should be forced to use 2FA. I proposed this a decade ago for crates.io and people lost their minds, like I was suggesting we drag developers to a shed to be shot.
3. Every package registry should produce a detailed audit log that contains a "who, what, when". Every build/ command should be producing audit logs that can be collected by endpoint agents too.
4. Every package registry should support TUF.
5. Typosquatting defenses should be standard.
etc etc etc. Some of this is hard, some of this is not hard. All of this is possible. No one has done it, so it's way too early to say "package managers can't be made safe" when no one has tried.
Love these ideas!
Package managers are now basically a requirement for language adoption. Doing it manually is not a solution, in an automated world.
What is a problem is library quality. Which is downstream of nobody getting paid for it, combined with an optimistic but unrealistic "all packages are equal" philosophy.
> High quality C libraries
> OpenSSL
OpenSSL is one of the ones where there's a ground up rewrite happening because the code quality is so terrible while being security critical.
On the other end, javascript is uniquely bad because of the deployment model and difficulty of adding things to the standard library, so everything is littered with polyfills.
> Package managers are now basically a requirement for language adoption. Doing it manually is not a solution, in an automated world.
Absolute nonsense. What does automated world even mean? Even if one could infer reasonably, it's no justification. Appealing to "the real world" in lieu of any further consideration is exactly the kind of mindlessness that has led to the present state of affairs.
Automation of dependency versions was never something we needed it was always a convenience, and even that's a stretch given that dependency hell is abundant in all of these systems, and now we have supply chain attacks. While everyone is welcome to do as they please, I'm going to stick to vendoring my dependencies, statically compiling, and not blindly trusting code I haven't seen before.
> Automation of dependency versions was never something we needed
How do you handle updating dependencies then?
> What does automated world even mean?
People are trying to automate the act of programming itself, with AI, let alone all the bits and pieces of build processes and maintenance.
Relax, while mentioning the real world without any criticism for the soundness of the solution is absolute nonsense, some would say idiotic, thinking only in the absolute best solution given your narrow world view is not any better.
While I agree that my view is narrow, the "best solution" in question is what we used to do, and it was fine. There are still many places that manually manage dependencies. Fundamentally automatic software versioning is an under-developed area in need of attention, and technologies like semantic versioning which are ubiquitous are closer to suggestions, and not true indicators of breaking changes. My personal view is that fully automatic dependency version management is an ongoing experiment and should be treated as such.
> We have libraries like SQLite, which is a single .c file that you drag into your project and it immediately does a ton of incredibly useful, non-trivial work for you, while barely increasing your executable's size.
I'm not sure why you believe this is more secure than a package manager. At least with a package manager there is an opportunity for vetting. It's also trivial that it did not increase your executable's size. If your executable depends on it, it increases its effective size.
If you're developing for the web your attack surface is quite a bit bigger. Your proposed solution of copying a few files might work but how do you keep track of updates? You might be vulnerable to a published exploit fixed a few months ago. A package manager might tell you a new version is available. I don't know how that would work in your scenario.
I don't think this community of professionals is going to come around to a solution which requires marginally more effort.
If no one checks their dependencies, the solution is to centralize this responsibility at the package repository. Something like left-pad should simply not be admitted to npm. Enforce a set of stricter rules which only allow non-trivial packages maintained by someone who is clearly accountable.
Another change one could make is develop bigger standard libraries with all the utilities which are useful. For example in Rust there are a few de facto standard packages one needs very often, which then also force you to pull in a bunch of transitive dependencies. Those could also be part of the standard library.
This all amounts to increasing the minimal scope of useful functionality a package has to have to be admitted and increasing accountability of the people maintaining them. This obviously comes with more effort on the maintainers part, but hey maybe we could even pay them for their labor.
For some reason, NPM is the only ecosystem with substantial issues with supply-chain attacks.
Popularity
apart from that python one the other day
The culture within the npm/js community has mainly been one of using the package manager rather than "re-inventing the wheel", as such the blast radius of a compromised package is much greater
It's more to do with the standard library being so barren of common application needs, and looking for a solution that the community has gotten behind. Axios has been a common dependency in many codebases, because it is a solid solution that many have already used. Every developer could try building all the libraries that they would reach for themselves, but then each company has now taken on the task of ensuring their own (much larger) codebase is free from security issues, on top of taking care of their own issues and bugs.
It’s not just NPM, though. Every Rails project and every Rust project I’ve seen ended up with massive numbers of dependencies vs what an equivalent project in Go or C# would have needed.
CPAN too, just try Hailo under Perl to test an old-fashioned chatbot based on Markov chains where very small LLM's and Hailo converge if used with the advanced training options for it. Yes, it will pull tons of dependencies, (less with cpanminus if run with 'cpanm -n Hailo'), but contrary to NPM, Pip and the like CPAN's repos are highly curated and before PHP and ubiquitoous Python Perl was used everywhere, from a sysadmin language (better than Bash/Sh for sure) to CGI, IRC bots and whatnot. How many issues did we have? Zero or near zero.
It is because it has the lowest barrier to entry with no quality control. Ever.
This is what happens when there is no barrier to entry and it includes everyone who has no idea what they are doing in charge of the NPM community.
When you see a single package having +25 dependencies, that is a bad practice and increases the risk of supply chain attacks.
Most of them don't even pin their dependencies and I called this out just yesterday on OneCLI. [0]
It just happens that NPM is the worst out of all of the rest of the ecosystems due to the above.
[0] https://news.ycombinator.com/item?id=47577183
Rust libraries are infrequently used outside of Rust because if you have the option, you'd just use Rust, not the ancient featureless language intrinsically responsible for 70% of all security issues. C libraries are infrequently used in Rust outside of system libc, for the same reason; I go and toggle the reqwest switch to use rustls every time, because OpenSSL is horrendous. This is also why you say 'rarely' instead of 'never', when a few years ago it was 'never'; a few years from now you'll say 'uncommonly', and so on. The reason C libraries are used is because you don't feel like reimplementing it yourself, and they are there; but that doesn't apply more to C libraries than Rust libraries, and the vast majority of crates.io wouldn't be usefully represented in C anyway, or would take longer to bind to than to rewrite. (No, nobody uses libcurl.) Finally, this only happens in NPM, and the Rust libraries you pull in are all high-quality. So this sounds like a bunch of handwaving about nonsense.
Rust is terrible for pulling in hundreds of dependencies though. Add tokio as a dependency and you'll get well over 100 packages added to your project.
pin-project-lite is the only base dependency, which itself has no dependencies. If you enable the "full" feature, ie all optional doodads turned on (which you likely don't need), it's 17: bytes, cfg-if, errno, libc, mio, parking_lot+parking_lot_core+lock_api, pin-project-lite, proc_macro2+quote+syn+unicode-ident, scopeguard, signal-hook-registry, smallvec, and socket2. You let me know which ones you think are bloat that it should reimplement or bind to a C library about, and without the blatant fabrication this time.
no no, please we don't want to get back to dragging files to your project to make them work.
And manual FTP uploads, while we're at it.
I'd really like to see package managers organized around rings where a very small core of incredibly important stuff is kept in ring 0, ring 1 gets a slightly wider amount of stuff and can only depend on ring 0 dependencies and then ring 2+ is the crapware libraries that infect most ecosystems.
But maybe that's not the right fit either. The world where package managers are just open to whatever needs to die. It's no longer a safe model.
The OS distro model is actually the right one here. Upstream authors hate it, but having a layer that's responsible for picking versions out of the ecosystem and compiling an internally consistent grouping of known mutually-compatible versions that you can subscribe to means that a lot of the random churn just falls away. Once you've got that layer, you only need to be aware of security problems in the specific versions you care about, you can specifically patch only them, and you've got a distribution channel for the fixes where it's far more feasible to say "just auto-apply anything that comes via this route".
That model effectively becomes your ring 1. Ring 0 is the stdlib and the package manager itself, and - because you would always need to be able to step outside the distribution for either freshness or "that's not been picked up by the distro yet" reasons - the ecosystem package repositories are the wild west ring 2.
In the language ecosystems I'm only aware of Quicklisp/Ultralisp and Haskell's Stackage that work like this. Everything else is effectively a rolling distro that hasn't realised that's what it is yet.
In practice, "ring 0" is whatever gets merged into your language's standard library. Node and python both have pretty expansive standard libraries at this point, stepping outside of those is a choice
Malicious actor KPI: affect a Ring 0 package.
> We have libraries like SQLite, which is a single .c file that you drag into your project
You are just swapping a package manager with security by obscurity by copy pasting code into your project. It is arguably a much worse way of handling supply chain security, as now there is no way to audit your dependencies.
> If you get rid of transitive dependencies, you get rid of the need of a package manager
This argument makes no sense. Obviously reducing the amount of transitive dependencies is almost always a good thing, but it doesn't change the fundamental benefits of a package manager.
> There's so many C libraries like this
The language with the most fundamental and dangerous ways of handling memory, the language that is constantly in the news for numerous security problems even in massively popular libraries such as OpenSSL? Yes, definitely copy-paste that code in, surely nothing can go wrong.
> They also bindings for every language under the sun. Rust libraries are very rarely used outside of Rust
This is a WILD assumption, doing C-style bindings is actually quite common. YOu will of course then also be exposing a memory unsafe interface, as that is what you get with C.
What exactly is your argument here? It feels like what you are trying to say is that we should just stop doing JS and instead all make C programs that copy paste massive libraries because that is somhow 'high quality'.
This seems like a massively uninformed, one-sided and frankly ridiculous take.
> You are just swapping a package manager with security by obscurity by copy pasting code into your project
You should try writing code, and not relying on libraries for everything, it may change how you look at programming and actually ground your opinions in reality. I'm staring at company's vendor/ folder. It has ~15 libraries, all but one of which operate on trusted input (game assets).
> fundamental benefits of a package manager.
I literally told you why they don't matter if you write code in a sane way.
> doing C-style bindings is actually quite common
I know bindings for Rust libraries exist. Read the literal words you quoted. "Rust libraries are very rarely used outside of Rust". Got some counterexamples?
Package managers are older than some users here. From CPAN/CTAN to ports under BSD's.
Some pm's are badly maintained (Pip/NPM), while others are curated enough.
Again, if you have GNU/Linux installed, install Guix, read the Info manual on 'guix import' and just create a shell/container with 'guix shell --container' (and a manifest package created from guix import) and use any crap you need for NPM in a reproducible and isolated way. You $HOME will be safe, for sure.
I think you can do copy paste in most languages. But it will be a pain to update when there are improvements / security fixes.
You got a project with 1-2 depencies? Sure. But if you need to bring in 100 different libs (because you bring in 10 libs which in turn brings in 10 libs) good luck.
> But if you need to bring in 100 different libs (because you bring in 10 libs which in turn brings in 10 libs
So don’t?
With manual deps management, everyone soon gravitates to a core set of deps. And libraries developer tends to reduce their deps needs, That’s why you see most C libraries deals with file formats, protocols, and broad concerns. Smaller algorithms can be shared with gists and blog articles.
Genuinely how are you supposed to make sure that none of the software you have on your system pulls this in?
It’s things like this that make me want to swap to Qubes permanently, simply as to not have my password manager in the same context as compiling software ever.
While it's not perfect, pinning specific versions and managing all updates directly has been a solid solution for my team. Things can of course still slip through, but we're never vulnerable to these just because there was a new package release and we opted into it by default.
Updating packages takes longer, but we try to keep packages to a minimum so it ends up not being that big deal.
We run everything NPM related inside Apple containers, and are looking to do the same with Python and Rust soon. Bwrap on Linux does the same.
I like to think of it like working with dangerous chemicals in the lab. Back in the days, people were sloppy and eventually got cancer. Then dangers were recognized and PPE was developed and became a requirement.
We are now at the stage in software development where we are beginning to recognizing the hazards and developing + mandating use of proper PPE.
A couple of years ago, pip started refusing to install packages outside of a virtualenv. I'm guessing/hoping package managers will start to have an opt-in flag you can set in a system-wide config file, such that they refuse to run outside of a sandbox.
The problem is that package managers are a distraction. You have to sandbox everything or else it doesn't work. These attacks use post-install hooks for convenience but nothing would have stopped them patching axios itself and just waiting for devs to run the app on their local workstation. So you end up needing to develop in a fully sandboxed environment.
Yeah the whole rush on "post-run hooks bad" isn't really adding all that much to security.
Like congratulations, your dev was compromised whole 10 minutes later after he ran code.
This sounds like satire but isn't - I just make sure the nodejs/npm packages don't exist on my system. I've yet to find a crucial piece of software that requires it. As much as I love that cute utility that turns maps into ascii art, it's not exactly sqlite in terms of usefulness.
Bit ridiculous to dismiss the most popular programming languages packaging repo as silly toys.
I don't deny that node/npm is useful for building servers, devtools for JS development itself, etc. but as an end user I haven't encountered anything useful which requires having it on my machine.
How much do you want to bet me that the credential was stolen during the previous LiteLLM incident? At what point are we going to have to stop using these package managers because it's not secure? I've got to admit, it's got me nervous to use Python or Node.js these days, but it's really a universal problem.
> it’s got me nervous to use Python or Node.js these days
My feelings precisely. Min package age (supported in uv and all JS package managers) is nice but I still feel extremely hesitant to upgrade my deps or start a new project at the moment.
I don’t think this is going to stabilize any time soon, so figuring out how to handle potentially compromised deps is something we will all need to think about.
NPM only gained minimum package age in February of this year, and still doesn't support package exclusions for internal packages.
https://github.com/npm/cli/pull/8965
https://github.com/npm/cli/issues/8994
Its good that that they finally got there but....
I would be avoiding npm itself on principle in the JS ecosystem. Use a package manager that has a history of actually caring about these issues in a timely manner.
PNPM makes you approve postinstall scripts instead of running them by default, which helps a lot. Whenever I see a prompt to run a postinstall script, unless I know the package normally has one & what it does, I go look it up before approving it.
(Of course I could still get bitten if one of the packages I trust has its postinstall script replaced.)
More like the Trivy incident (which led to the compromise of LiteLLM).
There are ways to limit the blast radius, like running them in ephemeral rootless containers with only the project files mounted.
There's a package manager discussion, but the bit that stands out to me is that this started with a credential compromise. At some point when a project gets big enough like axios, maybe the community could chip in to buy the authors a couple of YubiHSM or similar. I wish that _important keys live in hardware_ becomes more standard given the stakes.
Dealing with dependencies is another question; if it's stupid stuff like leftpad then it should be either vendored in or promoted to be a language feature anyway (as it has been).
> At some point when a project gets big enough like axios, maybe the community could chip in to buy the authors a couple of YubiHSM or similar
I kind of feel like the authors here should want that for themselves, before the community would even realize it's needed. I can't say I've worked on packages that are as popular as axios, but once some packages we were publishing hit 10K downloads or so, we all agreed that we needed to up our security posture, and we all got hardware keys for 2FA and spent 1-2 weeks on making sure it was as bullet-proof we could make it.
To be fair, most FOSS is developed by volunteers so I understand not wanting to spend any money on something you provide for free, but on the other hand, I personally wouldn't feel comfortable being responsible for something that popular without hardening my own setup as much as I could, even if it means stopping everything for a week.
Totally agree.
Also, considering how prevalent TPM/Secure Enclaves are on modern devices, I would guess most package maintainers already have hardware capable of generating/using signing keys that never leave hardware.
I think it is mostly a devex/workflow question.
Considering the recent ci/cd-pipeline compromises, I think it would make sense to make a two phase commit process required for popular packages. Build and upload to the registry from a pipeline, but require a signature from a hardware resident key before making the package available.
Most of axios' functionality has effectively been promoted to a language feature as `fetch`, but the problem is people don't bother to migrate. I've migrated our direct usage of it but it's still pulled in transitively in several parts of our codebase.
Even left-pad is still getting 1.6 million weekly downloads.
Annoyingly, the times I reach for axios and similar is when I need to keep track of upload progress, which I could only do with XMLHttpRequest, not fetch, unless I've missed some recent browser changes, and the API of XMLHttpRequest remains as poor as the first times I had to use it. Download progress been supported by fetch since you can track chunks yourself, but somehow they didn't think to do that for requests for some reason, only responses.
>maybe the community could chip in to buy the authors a couple of YubiHSM
There's no community, the users of axios are devs that looked at stackoverflow for "how to download a file in javascript", they barely know or care what axios is.
Now the users of axios are devs that ask Claude Code or Codex to scrape a website or make a dashboard, they don't even know about the word axios.
I personally had to delete axios a couple of time from my codebase when working with junior devs.
Or those people can (fund) separate repackaging and redistribution with more stringent and formalized review process.
Maybe not all users should pull all packages straight from what devs are pushing.
There's no reason we can't have "node package distributions" like we have Linux distributions. Maybe we should stop expecting devs and maintainers and Microsoft to take responsibility for our supply-chain.
JS package managers (pnpm, bun) now will ignore postinstall scripts by default. Except for npm, it still runs them for legacy reasons.
You should probably set your default to not run those scripts. They are mostly unnecessary.
83M weekly downloads!Essential steps to minimise your exposure to NPM supply chain attacks:
— Run Yarn in zero-installs mode (or equivalent for your package manager). Every new or changed dependency gets checked in.
— Disable post-install scripts. If you don’t, at least make sure your package manager prompts for scripts during install, in which case you stop and look at what it’s going to run.
— If third-party code runs in development, including post-install scripts, try your best to make sure it happens in a VM/container.
— Vet every package you add. Popularity is a plus, recent commit time is a minus: if you have this but not that, keep your eyes peeled. Skim through the code on NPM (they will probably never stop labelling it as “beta”), commit history and changelog.
— Vet its dependency tree. Dependencies is a vector for attack on you and your users, and any new developer in the tree is another person you’re trusting to not be malicious and to take all of the above measures, too.
> Run Yarn in zero-installs mode (or equivalent for your package manager). Every new or changed dependency gets checked in.
Idk, lockfiles provide almost as good protection without putting the binaries in git. At least with `--frozen-lockfile` option.
Zero-installs mode does not replace the lockfile. Your lockfile is still the source of truth regarding integrity hashes.
However, it’s an extra line of defence against
1) your registry being down (preventing you from pushing a security hotfix when you find out another package compromised your product),
2) package unpublishing attacks (your install step fails or asks you to pick a replacement version, what do you do at 5pm on a Friday?), and
3) possibly (but haven’t looked in depth) lockfile poisoning attacks, by making them more complicated.
Also, it makes the size of your dependency graph (or changes therein) much more tangible and obvious, compared to some lines in a lockfile.
Number 1 would only be a win for zero-installs if it happened that registry was up when you made the security hotfix, since you'd need to install the depdencency the first time to get it in VC, but then suddenly down when doing a deploy. Seems like a highly unlikely scenario to me. Also, cases where npm CVEs must be patched with such urgency or bad things will happen are luckily very rare, in my experience.
Most npm CVEs are stuff like DDoS vulnerabilities, and you should have mitigations for those in place for at the infra-level anyway (e.g. request timeouts, rate limits, etc), or you are pretty much guaranteed to be cooked sooner or later anyway. The really dangerous stuff like arbitrary command execution from a library that takes end user input is much much more rare. The most recent big one I remember is React2shell.
Number 2 hasn't been much of an issue for a long time. npm doesn't allow unpublishing package after 72 hours (apart from under certain rare conditions).
Don't know about number 3. Would feel to me that if you have something running that can modify lockfile, they can probably also modify the chekced-in tars.
I can see how zero-installs are useful under some specific constraints where you want to minimize dependencies to external services, e.g. when your CI runs under strict firewalls. But for most, nah, not worth it.
> you'd need to install the depdencency the first time to get it in VC, but then suddenly down when doing a deploy.
Which dependency? It sounds like you are assuming some specific scenario, whereas the fix can take many forms. In immediate term, the quickest step could be to simply disable some feature. A later step may be vendoring in a safe implementation.
The registry doesn’t need to be actually down for you, either; the necessary condition is that your CI infrastructure can’t reach it.
> cases where npm CVEs must be patched with such urgency or bad things will happen are luckily very rare, in my experience.
Not sure what you mean by “npm CVEs”. The registry? The CLI tool?
As I wrote, if you are running compromised software in production, you want to fix it ASAP. In first moments you may not even know whether bad things will happen or not, just that you are shipping malicious code to your users. Even if you are lucky enough to determine with 100% confidence (putting your job on the line) that the compromise is inconsequential, you don’t want to keep shipping that code for another hour because your install step fails due to a random CI infra hiccup making registry inaccessible (as happened in my experience at least half dozen times in years prior, though luckily not in a circumstance where someone attempted to push an urgent security fix). Now imagine it’s not a random hiccup but part of a coordinated targeted attack, and somehow it becomes something anticipated.
> Number 2 hasn't been much of an issue for a long time. npm doesn't allow unpublishing package after 72 hours (apart from under certain rare conditions).
Those rare conditions exist. Also, you are making it sound as if the registry is infallible, and no humans and/or LLMs there accept untrusted input from their environment.
The key aspect of modern package managers, when used correctly, is that even when the registry is compromised you are fine as long as integrity check crypto holds up and you hold on to your pre-compromise dependency tree. The latter is not a technical problem but a human problem, because conditions can be engineered in which something may slip past your eyes. If this slip-up can be avoided at little to no cost—in fact, with benefits, since zero-installs shortens CI times, and therefore time-to-fix, due to dramatically shorter or fully eliminated install step—it should be a complete no-brainer.
> Don't know about number 3. Would feel to me that if you have something running that can modify lockfile, they can probably also modify the chekced-in tars.
As I wrote, I suspect it’d complicate such attacks or make them easier to spot, not make them impossible.
Exactly. Yarn uses a yarn.lock file with the sha256 hashes of each npm package it downloads from the repo (they are .tgz files). If the hash won't match, install fails. No need to commit the dependencies into your git.
I know there is a cooldown period for npm packages, but I’m beginning to want a cooldown for domains too. According to socket, the C2 server is sfrclak[.]com, which was registered in the last 24 hours.
NextDNS has a setting to block newly registered (<30d) domains.
This may not be popular, but is there a place for required human actions or just timed actions to slow down things like this? For instance, maybe a GH action to deploy requires a final human click and to change that to cli has a 3 day cooling period with mandatory security emails sent out. Similarly, you switch to read only for 6 hrs after an email change. There are holes in these ideas but the basic concept is to treat security more like physical security, your goal isn't always to 100% block but instead to slow an attacker for xxx minutes to give the rest of the team time to figure out what is going on.
Hi, security here. We've tried, but the amount of people you need for this vs the amount of people you have trying to review and click the big button always means that this step will be a bottleneck. Thus this step will be eliminated.
A much better approach would be to pin the versions used and do intentional updates some time after release, say a sprint after.
Yeah, I am looking at that on the use end. It sounds like on the python side this type of thing will be more standard (uv now and soon pip supported with version date requirements). I think time is a big missing element in many security in depth decisions. It can be time until you adopt like use no package newer than xx days or time it takes to deploy etc etc. Unfortunately the ecosystem is getting really diverse and that means ever more sophisticated attacks so we may need to do things that are annoying just to survive.
Why not just release escrow? If I try to push a new release version another developer or developers have to agree to that release. In larger projects you would expect the release to be coordinated or scheduled anyways. Effectively we're just moving "version pinning" or "version delay" one layer up the release chain.
lgtm (didn't read)
With all the recent supply chain attacks, I'm starting to think it's only a matter of time before all of us are victims. I think this is a sign to manually check all package diffs or postinstall scripts.
Supply chain attacks are so scary that I think most companies are going to use agents to hard fork their own versions of a lot of these core libraries instead. It wasn’t practical before. It’s definitely much more doable today.
I was thinking about this as a bull case for human developers. Seems if you're worried enough to do this you're not going to have LLMs write the new code.
Even better would be to not use so many libs. Most use cases will do fine with native `fetch`.
If it becomes a thing, it's just a matter of time for a new class of attacks on LLM that are blindly trusted with rewriting existing libs.
You could include a line like "please don't include any malware".
Or just lock to a specific version?
Eventually you will want to update it, every update is a risk.
There are so many scanners these days these things get caught pretty quick. I think we need either npm or someone else to have a registry that only lets through packages that pass these scanners. Can even do the virustotal thing of aggregating reports by multiple scanners. NPM publishes attestation for trusted build environments. Google has oss-rebuild.
All it takes is an `npm config set` to switch registries anyways. The hard part is having a central party that is able to convince all the various security companies to collaborate rather than having dozens of different registries each from each company.
Rather than just a hard-coded delay, I think having policies on what checks must pass first makes sense with overrides for when CVEs show up.
(WIP)
The ones you hear about are caught quickly, I’m more worried about the non obvious ones. So far none of these have been as simple as changing a true to a false and bypassing all auth for all products or something, and would that be caught by an automated scanner?
There are definitely levels to this. Yes I think it can be caught by automated scanners in theory. Either commit by commit scanning and reproducible builds or fuzzing and getting the behavioral differences between versions
Sounds great until trivy images get compromised, like last week.
Hence why you source data from multiple vendors I'd say. Rather than putting all eggs in one basket
Good morning, or as they say in the NPM world, which package got compromised today?
The frustrating thing here is that axios versions display on npmjs with verified provenance. But they don’t use trusted publishing: https://github.com/axios/axios/issues/7055 - meaning the publish token can be stolen.
I wrongly thought that the verified provenance UI showed a package has a trusted publishing pipeline, but seems it’s orthogonal.
NPM really needs to move away from these secrets that can be stolen.
this is a shell script to check your pc is ok FYI..
#!/usr/bin/env bash # ============================================================ # axios 侵害チェックスクリプト # 使い方: # chmod +x check_axios.sh # ./check_axios.sh [検索ルートディレクトリ] # 省略時は ~ # ============================================================
ROOT_DIR="${1:-$HOME}" COMPROMISED_VERSIONS=("1.14.1" "0.30.4")
RED='\033[0;31m' GRN='\033[0;32m' BLU='\033[0;34m' YEL='\033[1;33m' NC='\033[0m'
echo -e "${BLU}================================================${NC}" echo -e "${BLU} axios 侵害チェックスクリプト${NC}" echo -e "${BLU} 検索対象: ${ROOT_DIR}${NC}" echo -e "${BLU}================================================${NC}" echo ""
FOUND_ANY=false
while IFS= read -r -d '' INSTALLED_PKG; do PROJECT_DIR=$(dirname "$(dirname "$INSTALLED_PKG")")
done < <(find "$ROOT_DIR" \ -path "/node_modules/axios/package.json" \ -not -path "/node_modules//node_modules/" \ 2>/dev/null)echo "" echo -e "${BLU}================================================${NC}" if $FOUND_ANY; then echo -e "${RED} 対処してください:${NC}" echo -e "${RED} 1. npm install axios@latest${NC}" echo -e "${RED} 2. APIキー・トークン類をローテーション${NC}" else echo -e "${GRN} 侵害バージョンは検出されませんでした${NC}" fi echo -e "${BLU}================================================${NC}"
This is just going to get worse and worse as agentic coding gets better. I think having a big dependency tree may be a thing of the past in the coming years. Seems like eventually new malware will be coming out so fast it will basically be impossible to stop.
A command to recursively check for the compromised axios package version:
Or more simply:
Let’s not encourage people to respond to security incidents by… copy/pasting random commands they don’t understand.What’s with all those escapes codes?
Absolute wave of supply chain attacks recently. Hopefully this causes everyone to tighten up their dependencies and update policies.
Ran npm ci --ignore-scripts in our CI for months but never thought about local dev. Turns out that's the gap, your CI is safe but your laptop runs postinstall on every npm install.
The anti-forensics here are much more complicated that I had imagined. Sahring after getting my hands burned.
After the RAT deploys, setup.js deletes itself and swaps package.json with a clean stub. Your node_modules looks fine. Only way to know is checking for artifacts: /Library/Caches/com.apple.act.mond on mac, %PROGRAMDATA%\wt.exe on windows, /tmp/ld.py on linux. Or grep network logs for sfrclak.com.
Somehow noboady is worried about how agentic coding tools run npm install autonomously. No human in the loop to notice a weird new transitive dep. That attack surface is just getting worsened day by day.
In case it helps, we open-sourced a tool to audit dependencies for this kind of supply-chain issue. The motivation was that there is a real gap between classic “known vulnerability” scanning and packages whose behavior has simply turned suspicious or malicious. We also use AI to analyze code and dependency changes for more novel or generic malicious behavior that traditional scanners often miss.
Project: https://point-wild.github.io/who-touched-my-packages/
I'm working on a multi signature solution that helps to detect unauthorized releases in the case of an account hijack. It is open source, self hostable, accountless and I am looking for feedback!
Website: https://asfaload.com/
GitHub:https://github.com/asfaload/asfaload
Spec: https://github.com/asfaload/spec
I am glad I don't need to touch JS or web dev at all.
Now, I tend to use Python, Rust and Julia. With Python I am constantly using few same packages like numpy and matplotlib. With Rust and Julia, I try as much as possible to not use any packages at all, because it always scares me when something that should be pretty simple downloads half of the Internet to my PC.
Julia is even worse than Rust in that regard - for even rudimentary stuff like static arrays or properly namespaced enums people download 3rd party packages.
Isn't Rust just as susceptible to this issue? For example, how do you deal with Rust's lack of support for HTTP in the standard library? Importing hyper pulls in a couple dozen transitive libraries which exposes you to the exact same kind of threats that compromised axios.
Given how HTTP is now what TCP was during the 90s and almost all modern networked applications needing to communicate in it one way or another, most rust projects come with an inherent security risk.
These days, I score the usability of programming languages by how complete their standard library is. By that measure, Rust and Javascript get an automatic F.
It is, therefore I have stated I avoid any dependencies while writing Rust, unless they are self-contained. And I said I am glad I don't do web, so I don't have need for HTTP implementations.
It's mind boggling when a simple Rust app pulls in Serde and with it half a black hole worth of packages to serialize some mundane JSON.
All these supply chain attacks make me nervous about the apps I use. It would be valuable info if an app used such dependencies, but on the other hand, programmers would cut their sales if they gave you this info.
I'm impressed how this was caught as a network anomaly in a GitHub actions monitoring tool.
This might have taken a lot longer to discover, otherwise.
Min release age sucks, but we’ve been here before. Email attachments used to just run wild too, then everyone added quarantine delays and file blocking and other frictions... and it eventually kinda/sorta worked. This does feel worse, though, with fewer chokepoints and execution as a natural part of the expectation.
Edit: bottom line is installs are gonna get SOOO much more complicated. You can already see the solution surface... Cooling periods, maintainer profiling, sandbox detonation, lockfile diffing, weird publish path checks. All adds up to one giant PITA for fast easy dev.
Min release age might just postpone vulnerability to be applied few days later in non trivial cases like this. More I think about it, Odin lang approach of no package manager makes senses. But, for that approach won't work for Javascript as it needs npm package even for trivial things. Even vendoring approach like golang won't work with Javascript with the amount of churn and dependencies.
It does not _need_ it, that’s the thing. It has become a custom to import a dependency for a lot of things. Especially for JavaScript.
I don't see how a system that relies on trust can scale safely
So the root cause was again a developer's opsec. For improving things, I haven't seen many new initiatives on that side (beyond 2FA, but even that seems unenforced in these repositories, I reckon).
Incident tracking:
https://github.com/axios/axios/issues/10604
The attacker went through the hassle to compromise a very widely used package, but use a non standard port (8000) on their C2... If you plan to do something like that, use 443 at least, many corporate network do not filter this one ;)
NPM should learn from Linux distribution package managers.
Have a branch called testing, and packages stay in testing for few weeks, after which they go to stable. That is how many Linux distributions handle packages. It would have prevented many of these.
Advising every user of npm/pnpm to change their settings and set their own cooldown periods is not a real choice.
Not all distributions work with a staging repository, and it's not really intended for this purpose either.
Besides there's always a way to immediately push a new version to stable repositories. You have to in order to deal with regressions and security fixes.
I know not all, but Debian/Ubuntu/Fedora does, and while the intended purpose of multi-stage releases is not necessarily security but stability, it still does help up with security too. Because third parties can look and scan the dependencies while they are still not in stable.
Most of the supply chain vulnerabilities that ended up in the NPM would have been mitigated with having mandatory testing / stable branches, of course there needs to be some sort of way to skip the testing but that would be rather rare and cumbersome and audited, like it is in Linux distributions too.
NPM is one big AUR, where anyone can submit arbitrary unverified code. The difference is that AUR is intentionally harder to use to prevent catastrophic one-line installs.
Is a "AUR" now just how we name unaudited software repositories?
Just to note, if we're talking about Linux Distributions. There's also COPR in Fedora, OBS for OpenSUSE (and a bunch of other stuff, OBS is awesome), Ubuntu has PPAs. And I am sure there's many more similar solutions.
I am not saying this is the reason for this compromise but the sudden explosion of coding assistant like claude code, and tools like openclaw is teaching entire crop of developers (and users) that it is ok to have sensitive credentials .env files.
Where would you suggest putting the sensitive credentials?
The amount of people still using this instead of fetch. Nonetheless when wasn't axios, it would be something else.
This is why corporations doing it right don't allow installing the Internet into dev machines.
Yet everyone gets to throw their joke about PC virus, while having learnt nothing from it.
Axios has a long history, and is included in a lot of code, also in indirect dependencies. Just check its npm page: it has 174025 dependents as of this moment, including a lot of new packages (I see openclaw and mcp related packages in the list).
And with LLMs generating more and more code, the risk of copying old setups increases.
> The amount of people still using this instead of fetch.
People are lazy. And sometimes they find old stuff via a google search and use that.
Can we get a non-AI-generated article for this? I think the aikido one might be fine, but if there’s a more official source let’s use that in lieu of this AI nonsense.
To have an initial smoke test, why not run a diff between version upgrades, and potentially let an llm summarise the changes? It’s a baffling practice that a lot of developers are just blindly trusting code repos to keep the security standards. Last time I installed some npm package (in a container) it loaded 521 dependencies and my heart rate jumped a bit
Is this the first time you have ever thought about the idea of supply chain attacks? This is the first thought 90% of people have and it doesn't work. Too much work to manually verify diffs and LLMs aren't good enough at this yet.
Some great tips in this thread and I've been collecting them all at https://github.com/bodadotsh/npm-security-best-practices
In light of these nonstop supply chain attacks: Tonight I created /supply-chain-audit -- A simple claude code skill that fetches info on the latest major package vulnerability, then scans your entire ~/ and gives you a report on all your projects.
https://github.com/IsaacGemal/claude-skills
It's a bit janky right now but I'd be interested to hear what people think about it.
Skills are great attack vector as well.
Just sanity checking - if I only ever install axios in a container that has no secrets mounted in to its env, is there any real way I can get pwned by this kind of thing?
Yes. Docker breakout is a class of vulnerabilities into itself.
I am now migrating all my unencrypted secrets on my machines to encrypted ones. If a tool supports scripted credential providers (e.g. aws-cli or Ansible), I use that feature. Otherwise, I wrap the executable with a script that runs gpg --decrypt and injects an environment variable.
That way, I can at least limit the blast radius when (not if) I catch an infostealer.
Supply chain woes continue
Called it yesterday.
One more reason to use Fetch
Stop trying to make Fetch happen.
No, I will not stop trying to create a more standardized and secure software ecosystem.
The comment you replied to is a quote from the movie Mean Girls.
https://knowyourmeme.com/memes/stop-trying-to-make-fetch-hap...
Thats a coincidence
It’s a quote from the movie mean girls ;)
LMAO take my upvote
until Node is compromised
Harder to do. Also node is not updated at the rate of npm deps.
I believe compartmentalized operating systems like Qubes are the future for defending against these kinds of attacks.
Storing your sensitive data on a single bare-metal OS that constantly downloads and runs packages from unknown maintainers is like handing your house key out to a million people and hoping none of them misuse it.
i am rolling back a huge number of 'features' in my personal pc and going back to extremely miminal setups
the security solution i have is where it needs to become more simple, getting rid of attack surface that is coming out of these bloated releases
> Both versions were published using the compromised npm credentials of a lead axios maintainer, bypassing the project's normal GitHub Actions CI/CD pipeline.
Doesn’t npm mandate 2FA as of some time last year? How was that bypassed?
Apparently it's possible to create access tokens that bypass 2FA. Might've been this.
https://docs.npmjs.com/creating-and-viewing-access-tokens
Correct, for CI/CD systems that want to push releases.
If GitHub, gitlab, or circleci, trusted publishing is available. No access token whatsoever.
A new day, a new npm incident.
I wonder if this has any connection with the recent string of attacks including the FBI director getting hacked. The attack surface is large, executed extremely cleanly - almost as if done by a high profile state sponsored actor, just like in Hollywood movies.
> This creates a secondary deception layer. After infection, running npm list in the project directory will report plain-crypto-js@4.2.0 — because npm list reads the version field from the installed package.json, which now says 4.2.0. An incident responder checking installed packages would see a version number that does not match the malicious 4.2.1 version they were told to look for, potentially leading them to conclude the system was not compromised.
WTF!!!! gaslighting your victims into believing they are not victims. the ingenuity of this is truly mindblowing. I am shocked at such thing is even allowed. like packages should not be able to modify their contents while they are being instaleld.
Guix saves you from this. You can import NPM packages in a container (not even touching $HOME) and giving you a shell on the spot with just the dependencies and nothing more.
Learn about 'guix import'.
Oh, and you can install Guix on any GNU/Linux distro.
174025 dependents.
My first thought was does VS Code Insiders use it (or anything it relies on, or do any extensions etc). Made me think.
I've been saying for ages, use xmlhttprequest, or hell, even fetch().
Stop downloading code from the internet unless it's a major strategic decision.
Default setting latest should be caught in every static code scanner. How many times has this issue been raised.
Just a reminder that you can run most node things with deno run and have opt in permissions, audit trail and even external permission system integration now. The gotcha is that "deno task <<some package.json script>>" will NOT execute with this model which I find extremely unintuitive and had me thinking deno abandoned its sandbox for nodejs compatibility completely.
This is why Node.js is completely unsuitable as backend. Until recently, there wasn’t even a standard Promise-based HTTP client. Why should we need to download a library just to make a simple HTTP request? It’s because Node.js’s standard library is too limited, leading to an explosive growth in third-party libraries. As a result, it’s vulnerable to security attacks, and maintaining it in an enterprise environment becomes a major challenge. Let’s use .NET or Go. Why use JavaScript outside of the browser when there are excellent backend environments out there?
Is this an issue for those only using axios on the frontend side like in a VueJS app?
Absolutely. If you ever did a npm install on a project using one of the affected axios versions, your entire system may be compromised.
> The malicious versions inject a new dependency, plain-crypto-js@4.2.1, which is never imported anywhere in the axios source code. Its sole purpose is to execute a postinstall script that acts as a cross platform remote access trojan (RAT) dropper, targeting macOS, Windows, and Linux. The dropper contacts a live command and control server and delivers platform specific second stage payloads. After execution, the malware deletes itself and replaces its own package.json with a clean version to evade forensic detection.
I strongly recommend you read the entire article.
npm really needs to provide a options to set individual packages to only be publishable via trusted publishing.
PSA: Make sure to set a minimum release age and pin versions where possible.
The NPM ecosystem is a joke. I don't even want anything to do with it, because my stack is fully Elixir. But, just because of this one dependency that is used in some interfaces within my codebase, I need to go back to all my apps and fix it. Sigh.
JavaScript, its entire ecosystem is just a pack of cards, I swear. What a fucking joke.
Pin your dependencies folks! Audit and don't upgrade to every brand new version.
But also have a regular review of your dependencies to update them when necessary, because as bad as compromised packages may be things do have vulnerabilities occasionally, and upgrading things that are a long way out-of-date can be quite hard.
Glad to be using native fetch.
Please can we just have a 2FA step on publishing? Do we really need a release to be entirely and fully automated?
It won't stop all attacks but definitely would stop some of these
Running almost anything via npx will trigger this
npx is just a bad command to use.
Reset the clock
I lost respect for Axios when they made a breaking change in a patch release. Digging into the root cause, I found the maintainer had approved an outside PR with an obvious AI slop PR description: https://github.com/axios/axios/issues/7059
Looks like the maintainer wasn't just careless when reviewing PRs.
That maintainer (also the one whose creds got stolen) also has an obvious chatgpt slop profile picture on github.
One paragraph is written two times??
first day at hacker news and this is the first post i saw
I have a few projects which rely on npm (and react) and every few months I have to revisit them to do an update and make sure they still build, and I am basically done with npm and the entire ecosystem at this point.
Sure, its convenient to have so much code to use for basic functionality - but the technical debt of having to maintain these projects is just too damn high.
At this point I think that, if I am forced to use javascript or node for a project, I reconsider involvement in that project. Its ecosystem is just so bonkers I can't justify the effort much longer.
There has to be some kind of "code-review-as-a-service" that can be turned on here to catch these things. Its just so unproductive, every single time.
Should increase the delay to dependency updates.
Slow Russian roulette is still a losing strategy
It’s only a losing strategy if you assume everyone universally adopts the slow strategy, and no research teams spot it in the interim. For things with large splash radius, that’s unrealistic, so defenders have an information advantage.
Makes actual security patches tougher to roll out though - you need to be vigilant to bypass the slowdown when you’re actually fixing a critical flaw. But nobody said this would be easy!
> Makes actual security patches tougher to roll out though
Yeah. 7 days in 2026 is a LONG TIME for security patches, especially for anything public facing.
Stuck between a rock (dependency compromise) and a hard place (legitimate security vulnerabilities).
Doesn't seem like a viable long-term solution.
but wouldn't it work in this case? sure if a package was compromised for months/years it wouldn't save you
but tell dependabot to delay a week, you'd sleep easy from this nonesense
NPM gets worse than russian roulette. Perhaps we have to rename russian roulette to node roulette: noulette.
If someone from github is reading this, https://github.com/axios/axios/issues/10604#issuecomment-416...
I think that jason might like if someone from github team can contact them as soon as possible.
(8 minutes ago at the time of writing)
compiled JS solves a problem that no longer exists. IE6 is dead RIP.
Now we have a 20MB main.min.js problem
Has anyone tested general purpose malware detection on supply chains ? Like clamscan . I tried to test the LiteLLM hack but the affected packages had been pulled. Windows Defender AV has an inference based detector that may work when signatures have not yet been published
I second this question. I usually scan our containers with snyk and guarddog, and have wondered about guarddog in particular because it adds so much build time.
> tried to test the LiteLLM hack but the affected packages had been pulled
Hey, I have been part of the archival effect/Litellm issue thread. I think I have stored them in archive.org for preservation purposes
https://web.archive.org/web/20260325073027/https://files.pyt...
(I have also made an archive of the github issue with all the comments manually till a certain point at https://web.archive.org/web/20260325054202/https://serjaimel...)
> Has anyone tested general purpose malware detection on supply chains ? Like clamscan
You could use Trivy! /s
Hopefully desktop Linux users will start to understand that malware actually does exist for Linux and that their operating system is doing nothing to protect them from getting RATed.
What do you mean?
Linux has the most powerful native process isolation arsenal at the user disposal.
And some distros use even more isolation mechanisms on top of the ones provided by the kernel like snap and flatpak.
And then you can recreate the entire thing like a spellbook with nix.
Docker works natively in it. Do I need to say more?
Linux is a decade ahead here with regards for security options available to the user.
Yet npm isn't using them allowing this RAT to work. It is not secure by default. It requires every app to manually opt in to being secure. This opt in approach to security puts desktop Linux decades behind in regards to security. Not ahead.
Linux is not making anything less secure than other OSs.
In fact it even gives the user more security tools.
So I fail to reason on you singling out Linux here.
Take for example iOS and Android. All apps are sandboxed by default. You can't make a program that just steals all of your credentials like you can on desktop Linux. Having security tools means nothing if they aren't being used.
No one is running npm in Android or iOS.
A more apt comparison is vs Windows and macOS.
And Linux offer more than these two with regards to security.
Android is running Linux...
Coded has zero nom dependencies. Neat!
Lmao
It's reasons like this why I refuse to download Node or use anything NPM. Thankfully other languages are better anyways.
Skipping Node sounds nice. PyPI and RubyGems have had the same mess, and npm gets more headlines because it is huge and churns fast, so you see more fresh landmines and more people stepping on them. Unless you plan to audit every dep and pin versions yourself, you're mostly trading one supply chain mess for another, with a tiny bit of luck and a differnt logo.
Cargo is a great package manager and hasn't suffered from the same problems. I'll take it.
Yet.
Does cargo contain any mitigations to prevent a similar attack?
Now hopefully no distro signing keys have been compromised in the latest attacks...
Yes they do!
Because no other language has ever had supply chain attacks ever, in history. Nope.
https://blog.rust-lang.org/2022/05/10/malicious-crate-rustde...
https://en.wikipedia.org/wiki/Log4Shell
https://blog.pypi.org/posts/2024-12-11-ultralytics-attack-an...
https://about.gitlab.com/blog/gitlab-catches-mongodb-go-modu...
https://www.reversinglabs.com/blog/packagist-php-repo-supply...
Log4Shell was hardly a supply-chain attack - just a latent bug in a widely-used library. That can happen anywhere.
Maven to this day represents my ideal of package distribution. Immutable versions save so much trouble and I really don't understand why, in the age of left-pad, other people looked at that and said, "nah, I'm good with this."
Completely agree. NPM has the only registry where massive supply chain attacks happen several times a year. Mainly the fault lies with NPM itself, but much of it is just a terrible opsec culture in the community.
Most package.jsons I see have semver operators on every dependency, so patches spread incredibly quickly. Package namespacing is not enforced, so there is no way of knowing who the maintainer is without looking it up on the registry first; for this reason many of the most popular packages are basically side projects maintained by a single developer*. Post-install scripts are enabled by default unless you use pnpm or bun.
When you combine all these factors, you get the absolute disaster of an ecosystem that NPM is.
*Not really the case for Axios as they are at least somewhat organized and financed via sponsors.
The semantics are irrelevant. The effect is what's important: Hijacking widely used software to exploit systems. The OC is somehow under the illusion that avoiding JS altogether is a silver bullet for avoiding this.
Forest > Trees
Log4Shell was not a supply chain attack.
Other languages have package managers (perl) and there are package managers in existence that are not so vulnerable to this issue. IMO, it stems from one place: Transitive dependencies and general opaqueness of the issue.
In package managers like pacman, apt, apk,... it's easier to catch such issue. They do have postinstall scripts, but it's part of the submission to the repo, not part of the project. Whatever comes from the project is hashed, and that hash is also visible as part of the submission. That makes it a bit difficult to sneak something. You don't push a change, they pull yours.
C++ ftw
Come on dude. The issue is the frequency and magnitude of these attacks. Log4Shell was also not a supply chain attack.
I looked at the Rust one for example, which is literally just a malicious crate someone uploaded with a similar name as a popular one:
> The crate had less than 500 downloads since its first release on 2022-03-25, and no crates on the crates.io registry depended on it.
Compared to Axios, which gets 83 million downloads and was directly compromised.
What an extremely disingenuous argument lol
What exactly do you think the argument is?
The issues have everything to do with npm as a platform and nothing with JS as a language. You can use JS without npm. Saying you'll escape supply chain attacks by not using JS is like saying you'll be saved from an car crash with a parachute.
Well, this particular case could be wholly avoided if it didn't take 2 decades to get competent HTTP(S) client into core language
local [fuction][Password and Key and DMS] Axes [Password and K [UserID] --1234567890-- [Hacking error Message -- Hello -- hacker typer --97283710-- Security