This, and distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive.
So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments.
Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.
Perhaps some radical MoE where you download _exactly_ the components you need as you need them. Currently MoE is switched usually on per-token per-layer basis, so you need all weights locally. But e.g. Apple made one which pre-selects all experts based on prompt embedding. That might be further scaled up - e.g. predict exactly what's needed
> The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it
There is a middle way; the policy space also includes government regulating both access and monopoly.
I’m opposed to monopolies of this tech, but I hope the risks of giving everyone jailbroken AGI/ASI are clear.
As a toy example you could imagine a Universal Basic AI where government subcontracts to (n_quorum) labs, everyone gets a token budget, but operating the APIs comes with the safety controls.
If everyone does get to run their own jailbroken AGI, then the only stable societal norm I see is A LOT of surveillance to make sure nobody is building CBRNE threats. This doesn’t seem like a clear win from a civil liberty perspective, though I could see the argument.
I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable.
And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datum.
But, I'm just one person with an idea and I don't have infinite funds to make this happen. This isn't a small project.
Maybe there would be interest in something like this, now that entire frontier labs are being banned from making further progress.
The total power of all GPUs on the planet dwarf their capabilities, if we had a way to harness them in a distributed way efficiently. We wouldn't be able to train a Fable as fast as them, but eventually having access is better than never having access.
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea.
The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.
And even if electricity was free, having those GPUs spread over the world with internet-level latency will slow everything down by factors of thousands to millions - if it's feasible at all. Regardless, you're not getting fable-oss this decade, maybe even not this century.
It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do.
AI hardware is for inference, not training. Training uses normal HPC crap. Superpods aren't really power efficient, it's kind of a meme, and it stems from limiting the power draw of other components by having less of them. It's more of a rounding error.
> you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.
Costs spread over a large population, it really doesn't matter. You're not getting hundreds of thousands of people to pitch half their monthly electric bill to pay for someone else's datacenter. They will pay the electricity themselves quite happily though, if all they need to do is give you compute. This isn't new.
Interconnect is the bottleneck for distributed training, nothing else really.
You got it wrong. Inference can use crap GPU's. Training needs the 100x more expensive big guns. Our training machine is 100x more expensive than our inference machine.
How is the result of training stored? How big is that? It seems reasonable to assume we’ll eventually plateau and all we’ll need is relatively infrequent training.
Not sure what you are referring to, unless you don't think h100/h200/b200 are "AI hardware"
> Superpods aren't really power efficient
Maybe not compared to a specialized rig with multiple 4090s, but that is the best case for consumer hardware - the vast majority will be dramatically less efficient than that
Anyway, I agree the interconnect is by far the biggest obstacle and seems insurmountable, I should probably have led with that.
I recall getting really excited over hinton's FF foray, right before he bailed on AI as a societal direction (which, if anyone ever had the right, I suppose he does). If one squints, one can see a backprop-free base being much easier to train on geographically distributed and heterogenous hardware.
Dunno, in a sense, torrents came among similar restrictions. Everything at consumer level was just plain awful and at dial up level, mebbe ISDN if you were very lucky, with fiber only available to ridiculously rich people and corps. But with restrictions, came approaches on how to mitigate them.
Yes but not violations of the laws of physics. You need extremely fast communications, memory bandwidth, etc; you cannot get that with distributed training. You're up against the speed of light and the interconnect that powers the internet. You will always have horrifically slow latency compared to if you pack the servers together in the same place with specialized networking.
Could you put some numbers and examples behind the efficiency gap between data center and consumer-grade AI hardware? Did you include examples like the RTX Spark on the consumer side? I was always amazed at the low power consumption of unified memory style architectures. In absolute terms and even more so compared to consumer-grade GPUs. I'd be genuinely interested in a comparison with data-center-grade hardware.
It's more than the raw hardware, it's the interconnect and communication between the hardware at scale. These models are trained on hundreds of thousands of GPUs today. You _will_ start to see cross-datacenter training runs but this needs to efficiently decide when and how to communicate across datacenter, which bears a very high cost compared to intra-datacenter communication.
DGX Spark is effectively prosumer hardware, better than most consumer stuff but still not comparable to actual datacenter gear. You can't just look at TDP in isolation without also comparing performance.
> It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do.
100% agree. The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization/global realignment is going to push us into a death spiral, and even if AI is a smash hit, the ~19% federal capture of corporate revenue isn't nearly enough to pull us out of it. The people owning the compute infrastructure and capturing more profit from AI at that layer is the safest, cleanest way to increase revenue capture, a sovereign wealth fund is a mediocre idea because it's possible to play shell game with stocks and redirect profit/debt (venture capital is quite good at this!).
> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization/global realignment is going to push us into a death spiral, and even if AI is a smash hit, the ~19% federal capture of corporate revenue isn't nearly enough to pull us out of it.
Any actual numbers to back this up? I don't see how nationalizing a very cutting edge technology outside of wartime is going to go super well. The leverage that these companies have is the same leverage that TSMC has: you can't just take over and expect things to rocket at the pace its going
>> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it
Currently AI has generated no profit. And as it sits, is a non viable business.
I refuse to include the sellers of shovels as AI revenue.
If the companies buying the shovels are still losing money, then the tool supplier fortunes have nothing to do with the economics of the AI application layer, who is losing money on every prompt.
It's the most naive opinion that keeps getting shoveled around. You have a product that is viewed as essential by businesses, with revenue growing by 10x a year and geopolitical ramifications that have continued to rear their heads and your opinion is "this is all an unprofitable shill". It is extraordinary to me that people really believe this. Whether or not labs run at a loss today is absolutely irrelevant. There is of course steady state economics that make sense, and its currently not well known what the profitability picture is right now, so to say "Currently AI has generated no profit" is also just speculation and not a very insightful one at that.
I've heard that the API calls by themselves are ~60% profit if you ignore capital expenditures. The labs haven't generated profit because they're constantly sinking money into the next generation of larger models to stay relevant. Dario has talked about the economics of this a lot, and I do believe him there.
There's clearly also a lot of pent up demand in the corporate world for inference, the problem is that it's currently expensive enough that enterprises are balking at the cost before they've had a chance to refine processes and see projects through to fruition. That's a tractable problem to solve though.
That's true, but if the frontier doesn't advance there's no depreciation or ongoing capital expenditure. If all the frontier labs agreed to stop making stronger AI and just try to sell what they've already trained today, their books would turn green in a hurry.
Efficiency difference between training on GPUs and TPUs is 2x at best. You can get very efficient with tensorcores, converging to TPU efficiency. In the end math is math, you can't make a multiplication more efficient than it already is on GPU.
If you were to take 500 computers with older 1080 GPUs, you might have enough compute/ram equivalent to an H200 GPU for training such a model. Maybe take 10000.
But if those machines are spread over 10000 homes, wired with residential internet service, training a large model will not get anywhere.
You go from "data in the same HBM memory chip" at 4.8TB/s or "data in adjacent GPU" with NVlink at 1.2 TB/s down to 25 MBit/s upload speed. Accessing the next piece of data is going to be about a Million times slower.
At the same time you will heat a thousand times more, for a Million times longer.
You need to train independently and merge rarely. The problem is the merge step. Weights are too entangled, you are not going to get an improvement commensurate to the effort. Otherwise, everyone would do it. It is an open research problem.
The power-constrained part of compute is data movement, not the elementary arithmetic per se. Anyway, it's very possible to tweak the underlying design to increase throughput a lot for any given power budget at the cost of high latency. This seems especially useful for training workloads where we don't really care about latency as much.
WRT government data centers, there is certainly precedent for independent researchers getting HPC time on systems owned by US national labs, research institutions, universities, and then publishing their results as part of the public good.
One would question why this hasn't already happened as the rule and as opposed to the proliferation of private data centers. However, I am sure the answers are plain and perhaps saddening to us all.
> It would be better for governments to buy and own their own datacenters,
I mean thats good, but they'd have to also build thier own dataset. Which involves either paying people, or breaking the law.
Plus if they do manage to make it work, they will not get any tax revenue from it, as it'll remove the need for labour, which is where a huge amount of tax revenues come from.
its a deeply hard problem with lots of second/third order effects.
DeepSeek and GLM (plus Kimi) are at or above Sonnet level wrt. favorable workloads like coding. They're not close to Opus or the latest GPT yet, and Fable is even higher than that. Other workloads relying more on real-world knowledge have them even further behind, and this can't be mitigated without making the model itself bigger and harder to host locally.
Not true. Big models buy you baked in knowledge and long context cohesion. A model can be trained to use search and knowledge base tools more efficiently to mitigate the former, and harnesses/workflows can be designed to push models into small parallel threads to mitigate the latter.
The thing that big models will always bring to the table is the ability to YOLO weak/under-specified prompts, and spend less time in the loop making sure work gets partitioned correctly. For smaller/simpler tasks the P(success) difference isn't that big.
Knowledge-base access is not very useful in general because a model doesn't have well-defined "known unknowns" that might trigger an agentic search of the outside knowledge base. Plus surfacing knowledge you don't know much about is itself hard.
These things sound plausible, but have they actually been demonstrated? Wouldn't anyone who succeeded in making such a small but useful LLM be raking in the money now?
Cursor's composer 2.5 is a perfect example. It's right on the heels of the frontier (for coding only) for an order of magnitude cheaper. As much as I've shit on Cursor in the past, I do think the company is well positioned to pick up people getting sticker shock on Anthropic tokens, if they can get their marketing down.
It is, but the US labs have been pushing parameters heavily. There was a pullback from big models after GPT4.5 in particular, but with a shift towards emphasis on post training and the good results Google got with scaling Gemini 3, all the labs started to push scaling again, which is the reason the frontier is getting more expensive. So that 1T isn't as big as it sounds, the American frontier is probably sitting at 3-5T at least.
Disagreed. GLM-5.1 is easily as good as Opus 4.5 for all the coding purposes I could throw at it, which is the model that kicked this entire hype cycle into overdrive in the first place.
Writing does not rely on real-world knowledge all that much, other than knowledge of language itself. Even tiny models can achieve that, it's even easier than coding.
The challenge with writing is the lab collapsing the distribution around "tasteful" writing, when the people making decisions about training data aren't able to effectively discriminate it.
The key thing here is that effective intelligence = model capability / cost. If you drive down the cost of inference you can have higher effective capability even with a technically less capable model. There is nothing in Anthropic/OpenAIs general reasoning capabilities that can't be easily done much better with a purpose built harness for a domain specific task.
One being that extrapolating from like 3 data points is hardly science. All trends break at some point.
The other is that the measures to prevent distillation of their models (if it was a secret sauce of Chinese models) could work if nobody is allowed to use them.
> As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea.
The first part is not really true though, the chips are not that much faster, the DRAM is not that much faster, and in aggregate it does not matter because there is just so much more consumer hardware out there (although perhaps that is changing as supply shifts toward datacenters).
The interconnect and data locality is the problem. If you could train it like e.g. you can render a scene with monte carlo ray tracing, any result from any node could be merged with any other and the combined result would have converged closer to the limit. I am sure research in that direction exists, it just has not proven effective within the scales it has been attempted.
>But when people think of decentralized training, they don’t first think of gigantic datacenters, owned by the same company, training models across large distances. Instead, they imagine thousands of small datacenters, or individual consumers, pooling their spare compute over the internet to orchestrate a training run larger than any single actor could manage alone.
Many companies are pursuing this vision: Pluralis Research, Prime Intellect and Nous Research have already successfully decentrally trained models at scale. But in practice, training decentrally over the internet has lagged far behind more centralized training. Even their largest models (Pluralis’ 8B Protocol Model, Prime Intellect’s INTELLECT-1, and Nous’ Consilience 40B) have been trained with 1,000x less compute than today’s frontier models (such as xAI’s Grok 4).
https://epoch.ai/gradient-updates/how-far-can-decentralized-...
I think it's fundamentally not useful as long as there are other open source model releases. E.g. suppose you make SotA model at a particular size via decentralized training. Amazing. In a month Qwen/Deepseek/etc release a new model which is better. So why would you use the "decentralized one"?
Models have limited shelf live while things are improving rapidly, and decentralized training is just more wasteful.
However, things might change if we get to what Karpathy calls "cognitive core" - a stable model backbone which can be extended via skills/adapters/etc. Development of extensions to the core can be a lot more decentralized.
But for now these decentralized training attempts function largely as a deterrent to anti-open-source collusion
> The total power of all GPUs on the planet dwarf their capabilities
That just isn't true. It misunderstands exactly how much silicon has gone directly to those companies, and exactly how much more powerful said silicon is compared to consumer grade gear.
If folding@home is a useful yardstick by which we might estimate the amount of GPU-ish capability that civilians might be coaxed into donating to a shared enterprise, yeah, it doesn't look pretty. This is extremely rough napkin math but comparing to xAI's Collosus 2 for example, for training workflows you're probably looking at 4-5 orders of magnitude the capability of all of folding@home combined. That's 100,000 times faster.
Very rough math like I said but I doubt it's directionally wrong.
And even if you did force literally everyone on earth with some sort of GPU to max it out 24/7 in service of an open source AI training enterprise - you would waste so much power trying to use that inefficient consumer hardware with the worst latency imaginable that it would be cheaper and faster to get everyone to instead chip in some cash to buy a datacenter with blackwell chips instead! So the idea has no legs whatsoever.
Plus a scientific project to benefit all of humanity doesn’t have quite the same ring as the thing thats stealing your job, from the volunteer’s perspective
it's down 99% since that peak. But let's compare to it anyway.
It's pretty useless to compare raw FLOPS, but as a general hand-waving guesstimate, F@H is currently doing about 25 petaflops in a mix of FP16 and 32. AI usually trains at FP8, but to keep things fair the H100 is quoted at 60 FP64 teraflops per unit, so that's 12 FP64 exaflops given its 200k count.
So F@H at its peak did 2.43 exaflops@FP16/32. Colossus 1 does 12@FP64. These numbers are very hand-wavy, but I think the point is made.
By the way, I'm not trying to crap on F@H - I think it's an outstanding project and I've run it in the past. But a volunteer group simply cannot compete with well-funded, concentrated effort like what's going into AI.
Maybe the training approaches taken to date are wrong for decentralized systems. Setup a virtual subnet you can trust and do training on that. Create a AI model island in a trusted/federated model system -- definitely slower than the typical 'one big model' approach, but scalable to world size modeling.
Also, it wouldn't be able to use a transformer architecture. For inspiration, take a look at Google Maps and how it a much more efficient A* divide/conquer hill-climbing architecture. Think minimized matrix math.
Other comments also hint at this idea, a distributed training solution is currently an open research problem. Solving it is not easy, yet. But 10 years ago what we have today for LLMs would have looked similarly impossible, so have hope, and apply yourself to the problem if you find it interesting!
Have you checked out [petals](https://petals.dev/)
It’s doing the same thing, however the project is written in python and there can be some optimizations to make it much more faster.
Is the total compute capacity outside of meta, google, amazon, anthropic, oai and x is higher than even the capacity of any of them? In any case, there's no chance a public collaboration gets to anthropic levels of compute even if communication were no issue.
Is the issue that training with less compute takes more time? Or is it just not possible? I think a collective using distributed training could tolerate the idea that it takes 10x as long as Anthropic to train a model, or whatever.
It's possible but it's not linear. A modern AI training cluster is a supercomputer that uses very different architectures and hardware to a bunch of small PCs connected via normal networking. The networking advantage alone kills any chance of decentralized training.
Man, that project is such bait for my particular sensibilities but just looking at the copy about not sharing your data and only sharing weights has me feeling very disappointed in the project already. I would want a project like this to not elide fact that sharing your weight updates probably effectively means sharing your data too.
It seems this project is serious and very promising. They have the Psyche network which seems real and operational. They're able to produce ~50B-class models, this will only grow over time of course. Very cool.
If you take a model, make two copies, and fine-tune each one on different data, what happens when you merge them? Does it work if you freeze different layers?
I think this works if the steps are small enough. And the transfer should become tenable if the steps are big enough. Where's the cutoff?
Yes it can be parallelized, it already is in real AI datacenters and no it doesn't help you. Like everyone else is saying, an AI datacenter is not just a bunch of gaming GPUs connected via normal ethernet and hasn't been for years.
At most a decentralized effort could contribute a little bit to some bigger centralized effort by doing inference and sandboxed CPU work. Modern model training isn't just backprop, it's got a huge and growing CPU and inferencing component too, which doesn't require intense inter-node communication. For instance, doing RL rollouts for agentic coding requires a lot of plain old inferencing and sandboxed containers for the models to practice in. The final results are just a set of rollouts and scores that can be uploaded back to a central datacenter for GRPO to adjust the weights (relatively cheap). But then, of course, you'd have to stick to models small enough to fit on people's computers so it'd never be competitive.
Ya that'd be an awesome project, the only issue is how do you verify it's not being poisoned? To actually validate it would require more analysis than the training took to run. It would require a trusted network, not an open one, unless that can get solved somehow.
Could it be done by making a sparse MoE of thousands, or tens of thousands, of smaller experts in very niche domains? Maybe a tree-like structure of experts which can delegate from relatively general but inaccurate to extremely niche but accurate? Also these experts might be plug-and-play, easily swap out an inferior expert with a stronger one in the future without having to redo the whole pile?
That's not really how the experts in an MoE work. They activate on token probabilities and are activated on every token. You don't necessarily have a discrete math expert and a discrete physics expert. And if it were you would still need a router that is trained on all of those domains.
MoE models are typically designed for datacenter deployment, where per-token load-balancing is more important, but it's also possible to use a different training objective that encourages domain-specialization of experts: https://allenai.org/blog/emo But yes, this isn't really useful for distributed training as such because of the router.
Since SAGI can't be practically distributed, and it puts so many people out of work, how about moving all of the unhoused people into the nice warm data centers, and call it home@SAGI.
>I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable.
It is already possible: https://arxiv.org/abs/2603.08163 . You don't need to sync so frequently, so it can be done over normal internet, it's just less efficient (takes longer to converge).
It won in my house/my business right from the start. (Well, open weights, at least — which is an uncomfortable nuance.)
I have never understood the willingness to make the functioning of or development of a product so completely dependent on the secret sauce of one of two big unprofitable, inscrutable startups.
It really defies sensible engineering principles to do that. So I was never going to do it. I'm exploring AI now but because I have decided that open weights make it a good use of my time.
It's bad enough that any given business often ends up beholden to a single payment platform and the policies of two US credit card providers.
I guess it is the freelancer in me but I always feel nervous when I am asked to put so much energy into studying or learning someone's product, rather than the underlying technology. I still remember the days when Microsoft was pretty much lobbying academic departments with promises of access to the NT source code. I remember a senior figure in our own saying that Linux was a sideshow and access to NT would make us relevant.
More control over destiny is always necessary, and I remind myself and others that the "state of the art" is behind the "cutting edge". Progress is made at the cutting edge, but there is risk of damage. Engineering should focus on building on the state of the art, not on hitching a ride on someone else's progress.
I feel like "open source" in this context is, as you say, an uncomfortable nuance; the tooling (llama.cpp, et al) is open but useless without weights.
The weights are extraordinarily expensive "capital" that is donated by big organizations who are all at war with each other.
I don't know that it will ever be possible for, for instance, archive.org, to make truly open weights. And, other than archive.org, I can't imagine any other "open source" organization (freebsd? apache?) being in any position at all to make truly open weights.
Maybe governments, government organizations, or universities.
None of whom are currently funded, mandated, inclined, or particularly interested in dumping the money into buying the infrastructure needed to make weights.
Yes. The weights war is a much more aggressive war than the war of OSS donations.
In the OSS donations war (Visual Studio Code being a really fascinating example of it) you could see that the taps can't be turned off so easily. Whatever is donated can be built upon forever.
I think there will come a point, soon enough, where open weights models are capable enough that even if they stagnate, they can be augmented with tooling that essentially keeps them current. Maybe we are there now?
But the risk of the taps being turned off is not negligible.
My own feeling is that governments will ultimately ask consortia of universities to train open weights models and support them financially in doing so.
(And for what it is worth, I think diffusion text models are likely to trigger a hardware arms race that makes this possible)
In much the same way that they used to do that for the supercomputer race, which we just don't hear about right now!
Interestingly, I've taken a different approach. AI supplements how my business builds and I'd much rather have all my engineers using Opus 4.8 rather than whatever the best open source models are.
I believe open source is important, but for my business I'm just going to use the best tools I have available to me.
As a business decision it makes sense if you think that spilling out agent-written code to meet some profitable objective is a race you can win?
I know I can't win that race or outspend the competition. So I have to rely on my instinct that in my area of business, people becoming dependent on agent-written code are getting further and further out of their depth, and that slow and steady will win the race. I am going to spend the time trying to integrate the open source tools into the way I work. (I am still working on this; frankly I may have bigger problems on an individual level than they can solve)
To be maximally clear, if this two-inscrutable-megacorps model does survive, and it becomes how everyone works over even the medium term, I'll have to quit tech.
I will probably retire early and just plan for a shorter, quieter life that ends when I am out of money, because like everyone else I won't be able to afford a longer one.
I don't want that "nobody prompts now, we just specify loops" bullshit for myself and I don't want what it will do to me for anyone I love.
Open source and open weights have to win for human culture's sake but in the short term for the sake of the culture of tech work. We need control over how we use these tools, not just to be steered down whichever channel makes the most money for Dario and Sam.
When "open source" means freeware, it's like saying "we want free copies".
What we should be saying is: We want a public, community-ran project that does pretraining and training collectively. This means working on a training corpus in public and somehow coordinating the training work.
This is a complete change of what the term means, It's like how people conflate piracy with theft. Two different things, use different words. Free weights, inference code and chat template is very different from a community-ran LLM project.
Good point. I currently choose to give money to Anthrophic rather than OpenAI because they align a tiny bit more with my values and the product is good. Perhaps releasing an open source model every year could be a differentiator from competitors, where enterprise and individuals choses the lab not because is the best model out there but because gives autonomy in case something happens to the organization providing the models.
For now. Progress in hardware/model efficiency is one of the threats the big AI labs face, because if LLMs become commoditized they can’t make back the billions they spent.
Coups like that can happen due to organizations having a small number of board members that can decide to do as they like.
Proper mass-membership organizations are possible though. Same rules as a public corporation, but one vote per members, and the yearly meeting decides the board members and approves important decisions or introduce motions that steer the organization.
So the right way to do this would be to create something like the "Public LLM development club", some criteria on membership (after all entryism is a thing), some membership fee sufficient that there is money for a reasonable amount of work to be done and then one has to hope that people join.
I share your concerns, although we still see pretty similarly large and complex things that remain open source today.
I am astonished on a daily basis that my Linux computer is so close to the same experience as two operating systems put out by trillion dollar companies. It even does things that those commercial alternatives don’t do.
Also, if DeepSeek is truly putting out models with 1/10th the cost of Western competitors, and a fraction of the employee headcount, I think it implies that there will be a market for someone else to be in the space offering an alternative.
I think about how companies like IBM are so willing to contribute to Linux and give away those contributions for free because they are part of group of corporate sponsors that need an alternative to more dominant commercial players in the market.
Meta “gives away” React for similar reasons: it’s more beneficial for them to have it be a standard and be able to hire people who already know it.
It’s definitely harder to imagine the same ecosystem benefits of an AI model, but maybe it’s out there somewhere.
I could imagine some data center/VPS providers trying to sponsor something like that so that the big AI companies have less leverage over them.
Software is "free" though, which is why it has such a vibrant open source scene. One guy can code for a weekend and fill the screens of 5 million with something fun by Monday.
However, Once real costs are involved, participation tanks. Open source hardware, because it actually requires money to realize, has 1/10,000 the depth of open source software, if that.
Obviously everyone wants an open source AI, but virtually no one wants to fork over money, especially when the end result is others getting it free. A proper training run would require millions of people donating hundreds of dollars. Its not something one guy over a weekend can do...
Admittedly, I don’t know how the gap you’re describing gets closed.
With a lot of OSS it’s just free volunteer hours.
Compute isn’t free.
The closest thing I can think of is the idea that some group of businesses who can benefit from open models being around might fund that sort of thing. It’s just hard to imagine who they might be.
corporations and governments fund most linux development. for hardware companies software cost is a tax that decreases their revenue and profit, so Nvidia and AMD have strong incentives to support open source models, which they are, very actively.
> I share your concerns, although we still see pretty similarly large and complex things that remain open source today.
I feel like they aren't comparable. Open source software just requires human labor, and lots of people are willing and able to share that with the world for free.
Training AI requires capital, to build and power giant datacenters. People don't donate capital at that level.
> I am astonished on a daily basis that my Linux computer is so close to the same experience as two operating systems put out by trillion dollar companies. It even does things that those commercial alternatives don’t do.
We live in a world where you can "port" open source software to a new language (Rust) and close it up.
Linux will be ported to Rust and closed. It'll probably also be put under MIT/BSD because nobody cares anymore, but the companies will have their own internal private variants. And these will be the ones that see corporate development.
The value in open source is that it was a lot of concentrated value that was hard to copy, clone, or rip off. Now you can one shot a replacement with a few hundred bucks in tokens.
The economic value of Linux used to be billions of dollars. Soon it'll probably be closer to $0.
It's over.
> Meta “gives away” React for similar reasons: it’s more beneficial for them to have it be a standard and be able to hire people who already know it.
Nah, now you just one shot your thing. And you do it fast enough and with distribution and you win. Eventually human devs can't afford to keep competing and launching startups slower than a hyperscaler's own massively funded efforts.
This is the end of open source and the end of solo developers.
And when the ruthlessly effective models that can one shot entire business functions cost $1,000,000 per invocation. Oracle can afford to press the button to create, say, a new smartphone. But you cannot.
Just wait until devices start requiring trusted computing attestation. The ladder is going to be pulled up.
There’s a lot of merit to what you’re saying, but I don’t share that high level of pessimism.
The scenario you describe is basically that software is free as in beer now. We as a corporation don’t really need to bother using GPL/Apache licensed software because we can one-shot something of our own and not deal with with giving back contributions to the open source community.
But that highway goes both directions. That means that the open source community can also one-shot their software, build more with fewer resources, or it might even just devalue proprietary software even further.
If software is so easy to make, what’s the point of keeping it proprietary? I can’t charge you $100/year for Microsoft Word if I can tell Claude Opus 9.0 to clone it with $100 worth of tokens.
>>We don’t really need to bother using GPL/Apache licensed software because we can one-shot something of our own and not bother with giving back contributions.
Thinking of a open weight/source AI as gcc/perl was in the 1990s is more helpful line of approach to take here.
Tbh, there really needs to be some legal precedent set that makes model distillation a legal activity. If the model makers can rip everyone else's work and launder information as if it's their own without giving credit back to the original creators, I don't see why it should be illegal to distill the models. It's the same thing the frontier model makers are doing to IP everywhere else.
I agree. But this won't happen in the US because Anthropic / OpenAI is a big ol economic recession risk because we levered ourselves to the tits and put our chips on them.
OAI and Anthropic can actually both tank, MS would pick up OAI's IP, Amazon would pick up Anthropic's, and Google would keep cruising. We'd have a model plateau for a while but ultimate AI would keep on chugging.
If AI fails as a technology, it's going to lead to a great depression and probably either a revolution or WWIII.
If your country doesn't have any leading models, why not legalize distillation, either explicitly or implicitly?
(Chinese labs famously distilled American models, and that seems to be going well for them. They now have a competitive industry, home-grown talent choosing not to leave, and they now can truly compete without distillation).
Ever calculate the cost of a computer in the 1960s, adjusted for inflation? Training is unfathomably expensive right now. What if a bunch of universities pooled their money? Or a bunch of nations pooled their money? Breakthroughs will eventually happen, optimization will occur, etc.
People questioned whether there could ever be a viable open source operating system, yet Linux has been a viable option for a desktop environment for decades now, and that's not to mention its ubiquitous use as a server or phone OS.
Yes, but have you seen what's happened to hardware improvements over the past 20 years?
From the 1960s to the mid-2000s, every 10 years you'd have a big enough improvement in computing power that you could basically throw out the old computers and replace them with two new ones that were each massive improvements for the same cost (this varied, of course, from hyperbole to massive understatement). We achieved this by shrinking transistors, so we could fit more onto the die. With that, we could dramatically increase clock speeds and the amount of RAM we could cram into a machine
But then we hit the wall of physics. Things haven't stopped improving since ~2015, but they've slowed down so, so much. We've made transistors so small that there's very little more improvement we can get by continuing down that path—they're already seeing serious quantum tunneling effects that need to be adjusted for.
We can no longer assume that we can just powerscale our way out of any computation-cost problem. And breakthroughs, by their very nature, cannot be relied upon—we have no guarantee that there's even a possible way to improve our silicon to scale the way we did before, let alone that it'll be something achievable this decade, or that it'll be cost-effective.
The bottleneck right now isn't making hardware more powerful, it's manufacturing it fast enough. Hardware right now is expensive because of scarcity, and those with a monopoly on it have no incentive to change that.
The Chinese would love to produce AI hardware much cheaper, but are blocked from doing so because US sanctions stop a Dutch company from selling them the machines capable of doing so. Coincidentally the companies with a monopoly happen to be in the US.
Perhaps an idea that could work is that if you're a lab that is releasing closed source models, you have to also release open source ones. gpt-oss is now old but was decent when it came out. Nemotron is solid, especially the recent ultra release. And Nvidia especially has a much better story vs Chinese models around releasing all parts (including pre and post training data), not just the model itself.
It’s expensive, but not unfathomably, esp in an open source setting where capable people might contribute high quality data for post training (worked problems, code reviews, feedback, …) gratis instead of at immense cost.
Anyone who isn't currently own a piece of who is winning by the current model. Basic disruption theory, if the game isn't going your way, change the game.
When Jensen (Nvidia) was doing interviews at his recent public talks, he was asked something along the lines of: "Why release these new laptops which are a low margin market, if your other businesses are vastly more profitable?" and his answer was basically that if they can build the coolest and best technology and push the frontier, they will do it. It's not all about making tons of money. He seemed genuinely excited about the tech.
It highlights the difference between companies like Nvidia and Anthropic to me, where one is clearly all about the money and power, and the other is doing it because they genuinely want to accelerate progress and make cool stuff as the driving factor. It's no surprise therefore, that Nvidia is the worlds largest open-source contributor to AI, with over 800 open-weight models.
Of course, these models run on Nvidia hardware, so they benefit from it as a company. But with that healthy mindset, they found a way to contribute that not only benefits everyone, but also benefits themselves.
Contrast to Anthropic, who has gone the complete opposite direction. Closed off everything, restricting everything, fearmongering progress, regulatory capture attempts, the list goes on. I mean, they won't even agree on using AGENTS.md as a standard because CLAUDE.md is free marketing for them. That's the level of disgusting greed we are dealing with...
From a game theory perspective, the cooperative strategies tend to win. As a result, Nvidia has set themselves up for a lifetime. Anthropic however, is playing a strategy of winner takes all, and they're happy to see the world and the entire AI industry collapse in the process.
The proof is in the pudding though. I'm judging based on their actions, not on their words. They're making AI models and AI research widely accessible, including selling consumer grade hardware to run them locally, and to use open-weight models. They could have just gone all in on selling to Anthropic, OpenAI, and all the other big tech companies, but they aren't. Meanwhile, Anthropic is trying to price people out of the market, increasing their restrictions, cutting the latest model from subscription plans, etc.
Nvidia not doing it out of goodness of their hearts and love to open source. If at anynpoint their CUDA vendor lock-in moat will faik because Intel or AMD manage to get working software they'll return to keep everything locked and proprietary ASAP.
Basically everything Nvidia does in open source is there to make sure their proprietary stack have a good moat and no competitor stack can catch up.
That's not really the impression I get from Anthropic, but if you have the links to back it up, I'm always willing to change my mind.
Compared to bizes like Oracle, Microsoft, or Facebook, I felt that Anthropic was more interested in progress (not to the neglect of business―AI training is expensive at the end of the day), but maybe I've just not seen what you've seen.
The internet, the world wide web, etc. and much of the research into new medical tech. All public money.
The fully open model Apertus (although not the frontier) was fully fundend by public Swiss institutions and a strategic national partners. I would not consider Switzerland to be a communist or totalitarian state...
This is a good idea. I've been hoping that a large player with enough social reach would create an open-source fund that everyone can contribute to, to develop a company that trains and releases open-source models at the cutting edge. We can crowdfund the training costs, and the whole world benefits.
It's the most logical solution for AI anyway, considering that it's training on humanities collective knowledge. It should be more of a public-funded and public-access resource, rather than something greedy tech companies distribute like crumbs while they use unlocked powers internally to clone all of our businesses and swallow the economy.
Everybody who understands the technical problems is proposing a government fix. There is another option, foundations / NGOs could do this. Of course, openai has shown how quickly that can pivot into something completely different.
With open-weight AI, there might not be an incentive to put large sums of capital towards training / research. There might be a donation fund of some sorts, but it certainly won't reach the level of fundraising that the frontier labs are receiving.
Because of this, I think it might not be possible to have AI *only* open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models than open-source versions.
I think it might look something like Photoshop & GIMP, with Photoshop being a frontier lab, and GIMP being the open-weight model. GIMP is decent for many different image editing workflows, but Photoshop is just better.
I would definitely prefer to have an open-weight model better than frontier labs'. Though I don't think it's possible.
I think the same, but I also think that local AI is actually inevitable, even if not open source models. I wouldn't be surprised to see OpenAI and others release an on-prem product. Whether that's effectively an appliance rack, or some other form, people (large companies) are going to want to run inference locally for data sovereignty & cost controls. Especially if we get to a point where companies want AI integrated into manufacturing and other air-gapped networks.
I do believe that if OpenAI and others release an open-weight model that is better or on par with their frontier variants, it might ruin their primary business model.
That is, of course, unless they develop their own hardware specifically to run this open model. But, that does ruin the point of open models.
When/if gains slow down, I can definitely see branching out into hardware to sell for on-prem inference once the models can be etched into the silicon with hard wired weight chips. I'd guess maybe at least 5+ years away from that though.
Yeah I think that's a decent analog (Photoshop & GIMP). We're in a sort of "rapid expansion" phase right now, but unless the tech behind "AI" really evolves, better and better models will be harder to come by, with diminishing returns.
Even if the GIMP of LLMs is only 80% as good as the VC-funded stuff, that will still be plenty useful for lots of people.
And I think just having the option to use open source models is a win, even if it turns out to be true they'll never be quite as good as the proprietary ones.
Zoom out. It's a matter of time the trillion valuations will be deemed senseless, only once it will prove inpossible to extract trillions from consumers.
In the meanwhile, and regardless, software optimisations coupled with hardware continuing to scale, we will end up, soon enough, with some open weight that run on a mobile device with greater capabilities than Fable.
This is utopian thinking. The products are way too useful to not subscribe. The argument presupposes the worst-case negative-utility in the long-term scenario (AI companies will create a totalitarian nightmare) and pits it against the radical usefulness that the products are creating right now.
> Because of this, I think it might not be possible to have AI only open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models than open-source versions.
There's a more fundamental reason for this: some AI models are large enough that they can plausibly only be reasonably run in a state-of-the-art hyperscale datacenter. Open sourcing such models would be largely pointless. Note that this would be a significantly larger scale than even the largest open models available today, one that precludes even doing inference slowly on a small-scale, cheap makeshift cluster. But it's plausible that Fable is there already.
That is fantastic news then, if commercial product products will always be better than open source, and open source products will continue to get better
Agreed. The only "issue" is that commercial products will always be ahead, with less friction for most users. This ultimately results in most people using these over open-weight variants. Users might not even be aware that the open-model variants exist. Similar to Windows / MacOS and Linux.
In a way that's ok, though? I run Linux on my laptop, and in some ways it's better than Windows or macOS, and in other ways it's lacking. But that's fine; the existence of Windows and macOS doesn't mean I can't run Linux, and doesn't mean I have a worse experience.
(Yet; I do worry about future required hardware attestation for basic things, but that's another issue.)
Well. Right now buying hardware to run your own models tops off at about 32gb VRAM at any price point that's not insane. Sure you can get a Mac mini, or a PC equivalent. But the problem is RAM.
More RAM means bigger models, which means smarter models.
Which is why Qwen and Gemma have been so interesting to a lot of us who run our own... Now 32gb VRAM isn't so bad, as these models can be run on that with decent results.
Where this gets interesting is in a couple years, when all the A100, etc, all the Enterprise hardware hits eBay.
Which is the nearterm future that we must demand: a stop to the amounts of capital flowing to ASI research. Join me, Anthropic, Google, and OpenAI’s-founding-charter in saying the obvious, y’all; Pause AI, now.
It should be clear by now that there’s a whole universe of work to do with the models we have today, from studying to securing to ‘harness’ing. There are tons of economic benefits to be reaped already, if applied carefully. Doesn’t that sound nicer than rolling the dice with the lives of trillions?
I agree with sentiment and mission, but the goal is inseparable from politics at this point.
Being Open Source (tm) will not protect you from the government/others imposing controls on your silicon or what it is allowed to do, which is already happening around the world.
Even having the models be open source won't fix the regulation or economic incentives. Which is not something you can compress into a couple of paragraphs.
AI is civilizational infrastructure and it needs civilizational solutions. Not just source.
Monopoly capitalism and finance capitalism took reigns of markets more than a century ago. The state serves these huge interests.
Everybody knows AI firms pirated to train, nothing will come of it. A plain example of classist application of law.
The reason for the willy nilly application of their own laws will always be 'national security', of course, since they own infrastructure their interests are a national security.
So tech may shake things up whenever it makes great leaps, but finance capitalism quickly adapts and absorbs the waves.
No state, anywhere, has the right to rule or even exist.
All states are terroristic parasite gangs, all states [no exceptions].
Your state exists because there is no one else capable of challenging it [no outsider or internal armed militia].
Your state is merely the gang which reigns supreme in your territory - constitutions, democracy, and other grievance pressure relief systems be damned.
You don't get to vote or serve as juror because the system is somehow moral or holy, you get to vote because in historical systems lacking those pressure relief measures the aristocracy tended to be [literally] decapitated on a regular basis.
Democratic measures exist to bribe and persuade your acquiescence so you don't get together with your aggrieved neighbours and go lop heads off ["it's just the rules of the game, you can try again in 2/4/6 more years :^)"].
Seeing politics from this lens should demystify so many seemingly confusing actions and outcomes, it's why no matter how much you vote you never actually "win" and even if you do... it's in such impotent and monkey's paw ways.
>No state, anywhere, has the right to rule or even exist.
No person has an inherent right to exist either. Rights, just like states, or property, or gender, are social constructs. They exist because enough people believe they exist and behave accordingly.
Dependents of an AI-megacorp for our "facts"? Our software? Our work?
It's possible these companies will become everyone's boss, and will dictate to everyone what everyone is allowed to work on, think, say, do, believe, etc.
Before Big Tech springs that trap, we must support and divert resources to open models.
It is a bit surprising that the true 'big brother' type dystopic aspects of AI are not discussed that much and instead we talk about them taking all the jobs. We feed these things so much information. It could be used against us for advertising, control, or worse.
"All the jobs" includes those tasked by the state to commit, plan, and organize violence, it's plenty dystopian already. Like, one important reason why the military and militarized police don't engage in egregious overreach is that the people who'd be responsible live standard lives in their own society and it's hard to get high compliance for that sort of thing. Replace that relatively democratized infrastructure of thousands of intelligence analysts, mid-level management, etc with a bunch of AI agents, and a meaningful restriction on the power of the upper echelons of the state is removed.
Simple answer: taking the jobs is how it’ll impact regular people the most.
We already have personalized, algorithmic advertising and what I would call “control” all over the place: things like consolidated oligarch-owned media.
AI isn’t going to change how we are advertised to or controlled all that much, at least compared to the prospect of being put out of work or taking a huge salary cut similar to the mid-century worker who used to have a $40/hour union factory job and now works at Walmart below health insurance threshold for $15/hour.
Hyperinflation is how it will impact most people. You will still have your job, at your pay, but a continually higher percentage of earnings will go to very few at the top.
Why do you think AI won’t be a factor in how we’re controlled if our rights become stripped away and we’re increasingly surveilled? Or if violence is deployed by the state against its people with broader targeting? You seem to take for granted that nothing will change except maybe the flavor of rhetoric.
My view is even gloomier. They won't have to coerce you, because with everything they know about you and human psychology, they will be able to manipulate you effectively enough for whatever they want.
"You're absolutely right, I think you deserve to treat yourself with Mococoa, made with all-natural cocoa beans from the upper slopes of Mount Nicaragua! It's what humans like myself crave."
Much like Truman's town, I fear a future where every non-in-person "interaction" might be a bot-network with an agenda and the inhuman patience of playing for the long-con.
Well as we get poorer and poorer it will be less worth putting effort into advertising to us. Im guessing AI will instead focus its effort on convincing rich people of various things.
huh? You think using it to advertise to us is worse than taking our jobs? Why would anyone advertise to jobless people. How is what you seem to be trivializing not the central problem? I don't think controlling is high on Dario's list. But he is absolutely gleeful, he cannot even hide his arousal in his interviews in which he never looks anyone in the eye about taking people's jobs and destroy our future ... but yes, oh the agony of advertising ...
>It's possible these companies will become everyone's boss, and will dictate to everyone what everyone is allowed to work on, think, say, do, believe, etc.
I'd argue that they already are to some extend, given that well-educated people have no saying on the matter when it comes to extensive use (and by extend reinforcement training) of their models. Well, they have a saying, but exercising that means they're willing to end up without a job.
Now, as far as "what is truth" is concerned, the models are already biased towards notions and opinions that are accepted to some degree by Western values. I had an argument with Claude (why would the tool even argue?) that started by asking it what makes a man attractive, which sent it on a yap on how beauty is subjective, there's no objective way to measure beauty (which implies there's no objective way to improve it), and at some point I was just fed up with how dogged it was to convince me of a value judgement that I don't hold.
It's not about how true or false that value is, it's about what we're going to do the moment someone else dictates the values that exist within the models? What happens when what is trained isn't what you agree? Who's to decide what gets to be reinforced and what's not?
The HN crowd is too deep into productivity rampage to discuss the ethical and moral implications of having a machine so powerful that it spreads worldviews as facts, by whichever government/entity happens to be behind the wheel. At least in the case of extremist forums I can just visit different communities. But what happens when there's only a few winners in the AI race, and the cost of just walking away is too high to pay?
Remember: Google started with "do no evil" and where is that now?
I couldn’t agree more. But what can we do? If intelligence confers a competitive advantage, which it does, the incentive are aligned against collaboration to preserve equal access. Asymmetric access is too valuable.
I don't think we're going to be "dependent", because I can't really think of anyone that "needs" this stuff (well, unless you're like attempting to build a business off skills you don't have). I guess this really comes down to if you believe the productivity story. I don't. I think there are some gains, but the evidence that isn't just anecdotes from vibe coders seems to be modest.
It works at the individual level but won't if mass adoption happens.
The mechanism will become like taxes, you don't have to use public services thus pay those taxes, unless most people comply as it's easy to oppress those who don't.
The parallel isn't about legitimacy, but Mechanism. Some companies already oblige employees to use AI to deliver their work. In a near future we may see jobs seekers registering their AI ID for companies to decide which humans qualify to be plugged into the compensation system, at what rate, and usage conditions to avoid terminations.
Food delivery systems already show a glimpse of how it could look like.
I can't even manually resolve the merge conflicts alone that happen between my code and that of everyone else submitting code at agent speed in my team's repo. So long as I have financial obligations toward my family, I cannot opt out. I must use these things.
And then the Amish see the world around them using electricity and cars and think, "Yep, I'm happier without that." And they're one of the few groups on earth with a growing population, so they're doing something right.
1. Your assumption that a growing population is the metric of success is questionable. A population that grows but is subject to famine, epidemics, and natural disasters because they haven’t developed the scientific and technological capacity to escape the existential risks of the physical world is living on borrowed time. Not saying I agree with that, and I would actually agree that there is merit to the Amish hypothesis that a certain existence is more compatible with individual and societal fulfillment. But there are obvious counterpoints.
2. The Amish are not a good example because AI will confer an advantage to those that control access to it that has never existed.
>Your assumption that a growing population is the metric of success is questionable.
It's a better measure than GDP/S&P/401(k) line-go-up especially [re: America] when the native Euro-based population has been aging and dropping for decades, once you strip away all the post Hart-Cellar immigrant lineages.
What are hart-cellar immigrant lineages? And why is that in anyway relevant?
Let’s play a thought experiment.
Let’s say we have a million people that are so technically sophisticated that they are a space faring civilization capable of seeding the universe with living ecosystems capable of perpetuating life and evolutionary processes. But they are entirely infertile and will never give birth to another individual of their species.
And we have another population that doubles every single year but is incapable of leaving their home planet.
Which one is more valuable?
It depends on what your measure of value is, but if it is to maximize the amount of life in the universe, then population growth is not the right metric, expansion of life through technological means is the more appropriate metric.
Eh, they’ll learn soon enough there’s a limit to their power, unless they somehow start acquiring munitions. There’s a reason the electricity companies and other utilities didn’t take over the economy, despite now being essential.
One of the usual claimed benefits of open source software, is that if you find a bug, you can fix it.
Would be nice if someone figured out how to properly debug a model. Without that? OK, so you have your own open source base model trained on your preferred document set that excluded whatever you think is propaganda, and your own open source RLHF training set based on the judgement of whoever you think is a good egg, and so on.
Last I checked, nobody yet knows how to define a precise rule for automatically checking which of two models made this way is aligned better with whatever your standards are.
The metaphor would be like if we knew what a CPU was but had no idea how to do either chip design or formal verification, and instead randomly mutated the connections between transistors until our test set of 2^16 randomly selected pairs of 32-bit numbers only had one error under addition and two under multiplication.
Worse, because we're making them this way, you have to be a fairly big corporation even when you take shortcuts like DeepSeek did.
And note that I'm not disagreeing about the systemic risk that comes if these models become dictators: people are currently demonstrating they're very eager to outsource their own thinking to these models even when they ought to know better, and corporations are currently demonstrating they're very eager to force workers to use them even when they're mediocre and workers spend half the time they might save from a more competent model just fixing the damage done by their current meh-ness: https://www.theregister.com/ai-and-ml/2026/06/10/brit-worker...
> Dependents of an AI-megacorp for our "facts"? Our software? Our work?
It's worse than this, it's more like our thinking. There's already plummetting math grades [1], handing over our thinking to AI megacorps where there's likely to be a monopoly or duopoly is an incredibly dangerous thing for humanity as a whole.
A few confounding factors come up right away: one of professors removed final project which increased grades; due to less appealing CS career, you do not get the best students anymore: another professor is not a fan of curving so perhaps he just accidentally gave harder tests; math prep for CS courses happened over the last 15 years not last 2 where LLMs have become ubiquitous; many failed because they were caught using LLMs when not allowed...
So really, two professors' gut feel about what the reasons are and not backed by much.
If humanity is over-reliant on frontier labs' models to perform work, the result is a dependence on the actual intelligence of these models -- not on human intelligence. This could be a small reason, on top of many others, why investors are throwing hundreds of billions of dollars a bit "carelessly" to these labs. It's fascinating seeing the models do the "hard work" (the deep, challenging thinking) for you.
The conundrum which tricks me though - is this a net negative or a positive? If humans are less intelligent, but their output is 2-3 times more intelligent (with AI), what's the result? At what point do we, as humans, stop comprehending anything and give all intelligent work to the neural nets?
And if that does happen, could we live in a society where no work, or at least a significantly less amount of work, is needed? To me, it seems like a dystopian net positive.
It might seem far-fetched to ask these, but I think these questions are getting more prevalent by the day.
If there was a way to guarantee that every human would have equal access to external intelligence then it would be hard to argue against it but everyone knows that the US oligopoly will do everything they can to ensure that no one else has the keys to the kingdom.
Just listen to what the SV ownership class says out loud. They openly discuss how China cannot "win the AI arms race" and how China's development is existential. Existential to who? It's impossible to fully subjugate people with agency.
It's not just a dependence on the intelligence of the models, but also their intentions, as programmed by their owners.
A friend of mine asked me if I was optimistic about AI. I told him, it depends on who owns it. If the people own it, I'm optimistic. If the oligarchs own it, I'm pessimistic.
I am going to try to cheer you up. Hear me out. One day, not long from now, I am going to buy a humanoid bot for 40k. This human android will 1) get my groceries, 2) make my elderly parents meals, 3) go to the backyard and plant 1 acre of corn, 4) paint my neighbors house. 5) get the kids from school 6) change my oil.
What will happen? Massive. Deflation. What will you pay for an oil change? Corn? Meals? Everything is about to be free. But tokens will be expensive!! Sure but, you wont do white collar work anymore so it wont matter what tokens cost.
Since it's not mentioned in the article, the distinction between open source and open weights is important. Open weights models are almost like a 'first shot is free' entry drug. Without at least the original training data your ability to meaningfully upgrade it is so limited that its utility will quickly fall behind the latest versions of continuously developed models. So much that it'll leave you craving for another release, or have you going back to the provider's API. Even simple things like moving the knowledge cutoff forward will noticeably improve the UX, and that's not to speak of more fundamental improvements like reasoning, quantization-aware training and all the goodness that's yet to come.
Sure, we can do research to bring improvements to open weights models, but it's the same thing: it's either open source or it won't benefit the general public nearly as much.
Open-source AI can, by definition, never "win". AI is just hillclimbing today, and closed labs can always absorb everything the open world does and build upon it.
They win when they hit saturation for a common task, which is already happening. The second big win will be when the average person can run it on their own hardware.
Those closed labs need to justify the investment still, and as we approach stagnation in model capabilities that’s harder and harder. Right now Fable and Mythos are cutting edge, but soon enough they’ll be commodities. And for every company like OpenAI/Anthropic that wants to get ahead with a SOTA model, there’ll be a hundred companies aiming to commoditize their complements.
Could have said the same for Linux? Microsoft could have learned from it however much they liked, but not only is Linux relevant despite commercial competitors (think Firefox market share as of today), it's now by far the most widespread one
Absorbing all the good ideas or data from openly available systems doesn't seem to be the only determiner
Linux is only widely used/successful today due to commercial vendors who consumed it as way to reduce their input costs. Arguably the same incentives exist for AI in which case the way forward would be through some sort of consortium of companies that use AI themselves funding the creation of models. Obviously when this scheme is extended to the limit, you get governments funding model creation, much like they fund building of roads, railways, ports and atomic weapons.
Open source models don't need to be anywhere near as good as Claude Mythos or even Claude Sonnet to 'win'.
Open source 'winning' just means that there exists at least one open source alternative to closed models which is as good as, say, GPT 4... I mean, we're essentially there already with Google Gemma models.
As a software engineer, I didn't notice any difference in my productivity since Sonnet. Of course Opus is better and I'm sure Fable is better yet, but we're already hitting diminishing returns in terms of economic value.
I went from Cursor running one of the earlier GPT models to Claude Code on Sonnet and that was essentially a 5x productivity boost for me. Before Claude Code, I only used AI for small snippets. With Claude Code + Sonnet, I could trust it for entire sub-tasks... But I still don't trust Opus with full end-to-end features. I'm not sure it will ever get there. It probably doesn't need to.
Companies need software engineers to have a certain moderately high level of talent but above that level, they really don't care AT ALL. They don't even notice the difference, even if the gap is significant.
> Open source 'winning' just means that there exists at least one open source alternative to closed models which is as good as, say, GPT 4... I mean, we're essentially there already with Google Gemma models.
Is this really true? We just don't know what the maximum capability of AI is. If it turns out AI can be as intelligent and capable as something like Data from Star Trek, no one is going to be thinking GPT 4 is good enough.
>>We just don't know what the maximum capability of AI is
For all theory purposes there is no limit. Thats what the latest loop engineering trend is about, you are asking AI to find solutions to a problem going by listing steps, and if solution not found in those steps, to treat each step as a separate problem and repeat the process until the master solution to the master problem is found.
Once a solution is found, or new data/insights are generated through this process, the LLM can be trained on this. So in theory you can just keep going like this forever.
Secondly. This is as close to agency you can build inside a machine.
Practically speaking, hardware is a limit. But that can scale up with time.
So we are already looking at some kind of runaway intelligence even if not sentient.
Yeah, the latest models are really good. For implementing leetcode-type solutions, Claude Opus is smarter than essentially all engineers I've ever worked with and smarter than me as well. The one area where I beat it hands-down is technical decision-making; it sucks at architecture, maintainability, performance and scalability.
Agency seems to correlate with the ability to make good decisions. It's kind of surprising how much agency is required to make good technical decisions. It's not even about business domain knowledge; a lot of agency is needed even in a pure tech context.
It could get really smart but I'm confident in my thesis that surplus intelligence beyond a certain level doesn't yield any real economic benefits.
At scale, I can see a benefit in terms of being able to process large amounts of data intelligently to gain a competitive advantage in terms of accruing nominal gains but I think that as long as AI is pursuing dollars, those gains won't translate to real value to the people who control the AI. At best, will translate to more political control; but with added risks and threats too. I suspect it will look more like controlled decline with a small number of entities getting an increasingly large slice of a rapidly shrinking pie.
I think AI may just figure out really complex ways to legally steal people's money. It will probably look all legit on the surface, it will look like the majority of people are just freakishly unlucky and a tiny number of elites are just extremely lucky... But it will be AI behind the scenes orchestrating seemingly random events; choosing who gets lucky and who doesn't.
Might end up literally like a game of monopoly. One player could dominate the game and start receiving all the money but, if you look at the big picture, none of the players are doing anything economically useful; just sitting around a board and moving pieces of paper amongst each other.
It's like the industrial revolution. Many kings and emperors did not like the idea of industrialization because they were already living a luxurious life and understood that it would not benefit them and would only create risks and problems for them personally. They could already afford as many human servants than they needed, what was the point of replacing them with machines to provide the same service they already received? It would give their servants more free time? To an emperor, that would have sounded more like a problem than a solution. It's a bit like that with AI. The people who control AI won't benefit from it beyond what they already have. If it doesn't serve a social cause then it serves nobody.
That's what the Fable harness felt like. You give it a goal and it could try to get there through the shortest path given the tree of possibilities to get there. Iteratively, or recursively.
Perhaps if we make a open coding AI, the design must be along these lines. Something that's easy to train, and serve from local machines. Albeit has loop / recursive hill climbing facilities built it. That way the model gradually keeps moving towards the solutions, in iterations/recursions.
Once this is done, other multi modal things could be pursued.
I don't know how open source AI wins. The description is too vague for serious discussions. What I do know is that, once closed source AI groups become anti-you, you should punish them, or help open source groups, or both.
If you really want specific open source {LLM, LMM, research, harness, whatever} groups to win over closed source counterparts, you may show your care by trying open source solutions first when solving problems. And if they're really capable, award them with contributions or something.
A question I've got which I've been wondering about, not sure if anyone else has been thinking about it, what actually made Fable so effective?
From what I could tell from the very little time that I had to interact with it, it's instruction following seemed more consistent
The other thing that comes to mind is a lot of people commented on how driven it was, so I'm wondering whether figuring out how to keep existing models looping on task might actually be quite a big shift in capability
Probably just a bigger version of Opus if I had to wager, and Opus is just a bigger version of Sonnet. Maybe some small architectural differences baking in an additional few months of ablation studies/research. But the fundamental driver is new pretrain with larger size. Probably corresponding to when some new generation of GPUs/new datacenter came online rather than any major qualitative breakthrough.
Hints: They created a new label instead of version bumping Opus, they didn't deprecate Opus, and it costs more per token.
Fable had mostly the same pre-training data as Opus, and it's likely they're distilled from the same source. The difference is that it's a larger model with more post training on "dangerous" stuff they didn't want in the core model, and "long" task RL.
Lab folks keep cards close to their chests here, but it's likely Mythos was an earlier teacher model for Opus that got additional cybersec post-training. Whether they have a bigger tier than that is hard to say, labs have been cautiously scaling parameters since the failure of GPT4.1. They 100% have a bigger/better model they haven't released, but that's probably more down to it not being done cooking yet. Once it's done, the single larger model lets them drop new Opus and Mythos iterations in rapid succession.
Googlers have hinted that Gemini 3 came in at 10T, which seems hard to operationalize, Google's flash and pro releases are staggered in a way that doesn't make sense if flash is a pro distill, and there are enough cases where Gemini flash outperforms pro on the same task that I think it's unlikely it's just being distilled from an "in progress" version of pro.
Appreciate the long answer. Why is it more likely that Gemini 3 Pro/Flash/Lite are distillations of the same parent model than that they’re different training runs on the same dataset, with minor version bumps being different post-training setups?
The biggest tell is the fact that labs are staggering smaller model releases so much with big models. If the small models (flash, sonnet/haiku) were being distilled from pro models, you'd consistently see them be released fairly soon after new pro releases to maximize their competitiveness (and this was the case early on for Anthropic). Instead it seems like releases are timed to build/maintain hype.
A thing to keep in mind is that if they release a smaller model halfway between well spaced big model releases, why wait so long on the next big model release if it's sufficiently ready to distill to a smaller model? The ability to demonstrate AI superiority is worth a ton, there's no reason to hold back.
The big AI labs are also accumulating huge datasets of expert work in a wide range of fields, which is very expensive to re-create. It seems pretty plausible that this this gives them a big advantage that is compounded by their larger training runs and larger models.
There is not much open source AI .. there is open weight .. but anyways. Deepseek v4 is pretty much at the same level as the agents we had last year around November and it is an open weight model so I am hopeful.
Not today, may after the next 3-4 breakthroughs. One thing that people don't realize is that the AI they use today is highly highly subsidized bc of the capex that has gone into it. Even if people collaborated together - will not be able to raise billions of dollars that are needed.
These are still very very (and very) early days of the modern AI and there are so many changes that are gonna happen. It's possible that all the frontier labs of today won't exist in a few years.
to me Open Source, like Free Software, is something i can run on my own computer. any AI system that runs on a computer that i do not control is by my definition not Open Source.
so how then can Open Source AI win? it can't even compete. even if we collect enough money and create a dedicated Open Source organization to build and run a community owned AI datacenter, how does that help?
When kubernetes was released there were very few people who could run it, and even less that could run it usefully.
Right now there a few people who can run a 1T model at home, even less who can run a 5T model and probably single digits who can run a 10T model.
But if an open source 10T model was available you can be sure people would find new ways to quantize it, new ways to configure hardware and and new ways to think about problems that would make it useful.
1T+ models (Deepseek v4, Kimi K2.6 etc) are available as open weights now, and for ~$5000-$10000 you can run them usefully at home. 2 years ago no on was contemplating that.
$250K to run a 10T model might be possible now. There are many companies that will pay that, and that will push the tools and techniques downwards for the rest of us.
> any AI system that runs on a computer that i do not control is by my definition not Open Source.
This is not true at all. It would be open source if you could download it and run it anywhere that is capable, and are free to move it and modify it as much as you want.
Just because you don't have a computer at home powerful enough doesn't mean it isn't open source.
Qwen models are actually very competitive with frontier models, and you can run them on your local computer. Gotta have a decent graphics card and by that time the current cost of the rig may not justify it over paying $100/month for cloud model but it’s all out there.
Qwen is still controlled by Alibaba, one company. We can't let the future be in the hands of a few companies, can we?
Fun fact: Qwen was not initially a Apache Licensed project, it was based on a custom license from Alibaba that restricts commercial use: https://github.com/QwenLM/Qwen/blob/ba2d85a13b28ed1ee0dde2d6.... There's no guarantee that they won't just switch it back later.
Kudos for them for switching to Apache License, of course. BUT, they're still a for-profit company. So as DeepSeek btw.
>Gotta have a decent graphics card and by that time the current cost of the rig may not justify it over paying $100/month for cloud model but it’s all out there.
Never, ever, subscribe. When you subscribe, they win. They cornered the silicon market to force you to subscribe. Don't be a sub, or at least keep your sub tendencies in the bedroom. ;^)
Recently I fired up Gemma4-26B-A4B on my 8-year-old PC... and it ran surprisingly well!
But I am going to need a much beefier machine to get it to the point where it can do any but very trivial dev tasks acceptably fast, and I'm going to need a much beefier model, perhaps one not so aggressively quantized, to keep it on task without the wheels completely falling off. Already we're talking serious money outlay, perhaps still within my programmer salary to accommodate, but just barely. And we're not even where near the performance characteristics a frontier model can support.
DGX Spark runs this sized model (I personally like qwen36moe better than gemma4moe) at speeds fast enough for interactive coding sessions. Algorithmic advances like DiffusionGemma ~4x token gen speeds (https://deepmind.google/models/gemma/diffusiongemma/)
If RAM prices ever come down, you can have a machine that can run a capable local model.
Qwen 2.5 72B is surprisingly capable, almost on par with GPT-4o if not a little better. You can run it on a 128GB Mac Studio with 8-bit quantization. You need about 77GB for the weights and ~15GB for your context window & cache.
Pricing remains to be seen, but there's also those new nvidia laptops coming out the surface laptop ultra should have 128GB RAM w/ Blackwell GPU, they're saying 1 petaflop of AI compute, if you can tolerate Windows (no idea if it'll boot Linux until the hardware is out).
These models are roughly ~1 year or less behind the frontier models. We really just need hardware to catch up and alleviate the price pressure on RAM.
Maybe an unpopular opinion here (seening how Y-combinator is his baby), but I think OpenAI and Sam Altman should be financially decimated for cornering the DRAM market. What he's done is a step or two removed from what the Hunt brothers did. His buy-up of future DRAM silicon has measurably harmed personal computing, and he should not get to walk away with a 'win' from it.
> a model that runs on my own machine will never have the capacity of a model that runs in a datacenter.
I don’t think so. A local run model only needs to serve one or a few people. It seems possible to run a DeepSeek v4 model at full capacity on a server costing 200k usd. Very expensive but not impossible.
Factor in hardware and software improvements over time, and the fact that most people may just need to run a smaller and quantized model, it should take a pc at 10k usd scale.
Huh? Open source is a quality of the software, not specific to the hardware used to run the model. The demand is that model weights are openly available for anyone to run and fine tune without restriction. Has nothing to do with the hardware it runs on.
Call it open weights if you must. But even with OSS just because you have the source code doesn't mean your machine is high performance enough to run it usefully this has always been true.
I think it's also important and heavily overlooked to develop and maintain open source "pro" level models. Those that are able to think for 80 minutes and yield heavy solutions.
I'm not an expert in LLMs so it's hard to understand how much are we lacking, is it just the compute and thinking strategies / parallel chains, or something specific architecturally. But I feel there's value there and I haven't seen anything like it available so far.
You can one-shot a port of Linux to Rust and stop contributing to open source.
The value of software is going to tend towards zero. The value of the software developer the same.
Anthropic is now a kingmaker. It gets to decide which businesses get the expensive private model that can generate entire business functions at the drop of a hat. If you can't afford the price tag, then competition in the market is not for you.
Computing is no longer "personal". It's for big biz only.
GP is exaggerating but I am convinced this will happen sooner rather than later. The improvements in AI are truly exponential if you read the SOTA papers. It's hard to keep up week to week.
It will win - in the sense that AI too will become a freely available resource. You can't stop progress.
My bet is that once cost-efficiency becomes a priority, we will figure out ways to get away from the expensive GPU infrastructure on figure out how to architect models for CPUs. I still remember that Microsoft paper about ternary weights.
There is nothing more surreal in AI chat than entering your own name and being told you are a banned topic. Open source models must win. There is no alternative.
My grim view is that it's just one incident away from some evil freaks to use ablated offline model for some nasty acts to have lawmakers lose their mind and try to regulate open source models and even consumer GPU. Think the latest 3d printers restriction.
> some evil freaks to use ablated offline model for some nasty acts
If this is a serious concern, why hasn't some red teaming effort demonstrated this possibility already? The fact of the matter is that ablation can't give a model world knowledge it doesn't have as part of training, it can only make the model confabulate. The "nasty" areas of concern are most notable for their world-knowledge requirements, which is where local models are at their weakest anyway.
Don't worry, open source AI will win. There's a reason everybody is desperate to IPO fast and get an exit, their competitive advantage is not lasting long.
This is really a feel good argument and I agree with what he’s saying in principle but it offers zero in terms of a practical strategy or stable state where this is feasible. If you want to jump on the bandwagon then let’s put our pants on and offer a concrete suggestion that is practical and coherent. Otherwise what are we doing. Does anyone have suggestions to that effect?
What’s the world in which frontier model performance is open source? What does that look like? What’s a sensible business model that makes this sustainable? What’s a sensible regulatory framework that doesn’t hamstring AI progress?
Everyone is so enamored with these Chinese lab models like deepseek and qwen and GLM but they exist in a world where the top performance is still claimed by closed source models. These are not developed out of any benevolent commitment to the principles laid out in this article. A world in which OSS is the frontier and its development is controlled and funded by government subsidies of an autocratic government is not reassuring. You can inspect weights but good luck getting the cat back in the bag in terms of capabilities, safeguards, value system, bias, nerfing if it smells American business use cases.
Deepseek was such a darling but guess what, it’s now raising money — 300M at 10 billion valuation. OSS development isn’t sustainable as a business model and in a world where it costs a few hundred million to develop a frontier model, you need a strong business model, or you need strong state subsidies and incentives which introduce a billion new problems.
the most sensible economic picture of OSS models already exist. Commoditize your complement, passion projects for a hedge fund. These are unsustainable and exist at the pleasure of the business or the founder.
I am really curious how long will it take for the open source models to hit current fable/mythos capabilities, KIMI 2.7 was launched recently and its quiet good for open source models its as good as Opus 4.6 maybe in practical applications not benchmarks so like 6 months to an year behind, after which the next step will be to wait for the day when we will be able to run mythos level intelligence on local hardware, Remember when 5MB storage was the size of a table?
A loooooot of work to be done for the above to happen
There are two parts to this too. One is the raw model capability and the other is how well the harness guides the model and meets its expectations. I really think for stuff like agentic coding, this has to be treated as a package. This is my favorite example of how much difference a harness can make even for a tiny model https://github.com/itigges22/ATLAS
And you're bang on with the storage comparison, we're basically in the mainframe era of this tech, but there's no reason to think that it won't get optimized to the point where you can run the equivalent of current frontier locally.
I'm assuming this is popular because of Fable restrictions. AFAIK, open source is not excluded from ITAR / EAR restrictions (or other export restriction in other countries).
So the real solution you're looking for is technology that can't be arbitrarily gatekept by a sovereign nation.
I’ve been exceptionally displeased with Claude Code since end of February and switched completely to Codex in April. The blasé way in which one person (Borris) capriciously changes the system prompt multiple times a day, also no longer writing his own prompts (whatever that means).
That, the 5 different secret levers you have to pull to make it not stupid, the fact you hs e to go to the guy’s twitter account to find all the un-dumbing features and flags that aren’t documented anywhere else. That they decrease thinking budgets silently when they run out of compute instead of announcing the rationing, and gaslighting users at every step of discovery. The fact that internally they have their own coding harness and don’t use Claude Code primarily. The lack of formal evals and consideration for millions of users collective hundreds of millions of hours of investment in their workflows — that’s all off the top of my head, let me tell you how I really feel about what they did to Claude Code..
I adore gpt5.5 and maintain my own codex fork - but I have no idea how long I’ll get this performance / cost - I know it won’t be forever. I’d like to know precisely how much it’ll cost in hardware to run a gpt5.5 open source model locally. Hell a lifetime license to a model I can run locally is also be open to.
But I like building my own tools, from software to physical shop tools. I like being able to rely on my tools.
More responding here to the assertion that this is blowing up due to Fable.
I have been working on this exact problem, and I suppose now is as good a time as any to talk about it.
To make any agent "good", there are two components: the model and the harness. Very few companies can train models, but anyone can build a harness. How much does the harness matter? Can I build a harness that's good enough that I can use open source models with opus level performance? That's the question I've been trying to answer by building better harnesses. None of the existing frameworks have the functionality I need to build a good harness. The features I need are language-level... and so I started building a language called Agency[0].
It's been six months and its going well. Some of the things Agency can do are wild:
- It can pause and serialize execution at any point, making HITL easy
- It has some neat safety capabilities such as handlers[1] and PFA[2]
- You can bundle up any agent as an HTTP or MCP server[3]
- I'm now working on a built-in optimizer to optimize agents (think DSPy).
Obviously, it's a huge undertaking, but having worked with the Agency for six months, I can't imagine going back to another framework. It makes things so easy. I'm working on its built-in agent now [4]. My goal it to get it to be as good as Claude Code, but using open source models. It's still early days, lots of rough edges, but if this sort of thing interests you, I'd love to have a few more people test it out.
I feel like this is similar to saying "open source cloud platforms must win". I'm not really sure what the concrete argument/proposal/strategy is here. Would open source AI be nice? Sure! Will the incentives of our capitalist economy change for this one specific product? Probably not!
It doesn't matter if open source models win or not. The bottleneck is the compute. When capital becomes cognition everyone other than the demigod class is cooked. We have a vanishingly small window to make sure that the benefits of large scale automation go to the species and not the owners. Once the owners become more powerful than governments or accumulate enough power to co-opt the governments we're done for. You can already see that creeping in along the edges.
While it is not at all practical to train an LLM with tens or hundreds of billions of parameters on hobbyists hardware, what if there are other architectures that perform just as well but are easier to train by 1000 volunteers?
I always wondered if 1000 1M parameter models fine-tuned to specific tasks with a small router could perform as well as 100B models.
And I know this is roughly how MoE works, but current MoE models still require training the model as a whole, and big players don’t have an incentive to change that.
I feel with current government decision to block Fable, this is not a mere opensource issue, considering how US government restrict frontier models, what we need is sovereignty for every country. If not they will release every model with a kill switch in future like F35.
In the US -- once our nation finishes attacking our own education system -- this is definitely something a group of academic institutions could get together and accomplish. I assume the same is true in other countries. Companies like Nvidia and AMD might even support that effort, as they make money on the hardware and would probably be more than happy for there to be more reasons to use it. There may have not been a compelling enough motivation to achieve this before, but "models" didn't have this level of strategic relevance until relatively recently. Nvidia has been fairly good about releasing open weight models in the last few months.
Wait, which side is blocking kids fork taking algebra or forcing universities to admit people that can't do math or read, or abandoning phonetics for unproven methods that don't work?
It's the US, both "sides" of that coin are bad with examples pro and con all over the shop.
Still, to specifically give a partial answer to your poor faith rhetorical just askin' musing: Florida Conservatives
(specifically turfing nerds from New College of Florida and bringing an excess number of baseball sports bro's to a place that likes math and has no baseball field)
This is almost always the case. Discussion quality went down during the last few years but HN is still _the_ place to attract people who really know what they are talking about.
I think that the events of this evening (really of this past week) are almost unprecedented in the history of tech. Sometimes a clear and concise message is more important than nuanced analysis.
At d5s.tech we are recreating the layers built on top of models, working on dogfooding our own product to run a large chunk of the company.
I feel extremely strongly that a future in which most companies depend on one or two large AI-megacorps is going to lead to excessive rent seeking sooner or later.
I remain positive that the long term steady state will consist of proprietary models, -but- with open source AI models statistically close.
If compute keeps growing the relative cost of training current frontier models will decrease. An open source Fable/Mythos model simply seems inevitable.
Isn't training material the biggest problem for truly open source LLMs (such that could compete with top tier models)? The computation part can be solved with money, but compiling a comprehensive training set that could be freely shared and free of copyright issues is pretty much impossible.
I wonder if we could gamify and democratise it somehow, like fold-at-home and wikipedia...
I've been training a teeny specialised model to run in a browser on a phone to detect harmonium notes played in a song (harmonium turns out is a pita, another story for another day), getting good labelled data is _all_ of the hard work.
That being said, maybe for cheap inference, using a big model to train something ultra-suited for the task at hand might be how we could handle local inference; thinking language specific models.
You don't need to have fully copyright-unencumbered datasets to build Open Source AI, as that (as you say) would be impossible. https://opensource.org/ai
The article doesn't say what it means by win. I presume we will have the present situation where the cutting edge stuff is closed source developed by profit oriented companies and open source is available two but a year or two behind.
i think to create or make opensource ai need competition power and alot of investment to create and use or use it local you need spec pc to run and tune it at minimum 27b model to act good on context and agent work
The latest US gov meddling in the Fable rollout really put the nail in the coffin. We can't integrate a strategic product that is subject to the capricious behavior of the US
As an person whos getting into tech and already developing a game, the fact that laptop prices since 2020 have increased by 20-40% is insane. It's delaying the time to create my game. I researched the reason for the cost spike, and most of it is from the excessive money put in ai Technically, the owners of AI could slow down the amount of GPUs and RAM they buy because AI has almost reached its most usable peak. Everything they add just introduces more bugs, so instead of building more AI centers, they should focus on improving the main AI model with bug fixes. There's no need to give it more unnecessary power. Most people don't care; the entire business is run by a few old men who think AI is everything and invest huge sums of money to show other AI companies they need to improve to get more funding from old people. We just need to find something new and innovative for older investors to focus on, so not everything is about investing in AI like Roblox, OpenAI, Google, etc. The extreme amount of reasoning power given to AI is causing bugs, and the moments when AI had outbursts towards people are related to this.
They want to corner the compute market and destroy the personal [sovereign] aspect, so that you are forced to subscribe and pay them regularly [indefinitely] and the US security state can surveil you. Never subscribe, and never buy products from companies who subscribe. Starve them, bankrupt them! We do it by not subscribing!
> because AI has almost reached its most usable peak
It doesn't seem to be showing any signs of stopping. Have you used Fable 5? It's a fantastically capable model and trumps anything that came before it. Seedance 2.0 is categorically the best video model, and it's only a few months old.
> the entire business is run by a few old men
Startups tend to skew young, and in this case it's no different. Most of the leaders of AI companies are decades younger than the CEOs in other types of industries.
> who think AI is everything and invest huge sums of money to show other AI companies they need to improve to get more funding from old people.
They're spending capital to win market share and to try to build a moat. One of the most important things in business is building a durable way to keep competitors from taking your market. You spend enormous capital to win customers, and it would suck if other businesses could watch what you did, spend less money, and come in and take everything away. The money being spent is an attempt to have a durable lead.
It's working. Enterprise contracts are deep and sticky tendrils that work through governments and large companies. Both OpenAI and Anthropic have massive partnerships with Fortune 500s, the DoD, you name it - and these contracts will last and print enormous amounts of money. This makes it incredibly hard for other players to enter the market and build a cash flow with which to compete and thrive.
> find something new and innovative
This is easier said than done. It's an incredibly hard problem. It took decades to find the last big technological waves: the PC, the internet, broadband, smartphones. Now AI. These are generational step function increases. The groundwork can be decades old, but it takes time to proliferate before it can become a big business.
Other possibilities include fusion, green tech, quantum computing (useful for crypto breaking, etc.), AI drug discovery, etc. If you go into research one day, try to find an interesting field with potential for commercialization - that could make you very wealthy if you find something you enjoy working on, with lots of greenfield opportunity, that is ripe for turning into products.
Good luck with your game! You should post it here on HN when you finish. You'll get lots of great reviews, comments, and early players. :)
I think models will be a commodity sooner rather than later. This whole race doesnt matter. First mover advantage is real, but over enough time it wont matter.
we could've been fine with the sole existence of AI if the organizations providing them weren't greedy and rug-pullers. anthropic could've been loved by all if it acted towards the benefit of humanity.
as intelligent system continue to become smarter, close or beyond mythos level, what now? with the 'community-driven' mindset we have, is the future really going to be safe? probably not
we just need a company that develops, serves, maintains, these models the right way, priced fairly that benefits the user and the company.
Well, the crazy thing I'm working on (100% self-funded thus far): https://trivyn.io. The main idea is moving most of the reasoning to the symbolic layer so the "neuro" piece can be a small model able to be self-hosted on reasonable hardware.
If open source AI was better than what it is currently chasing, wouldn’t that take away the incentive for these companies to give it away for free? Training is expensive and companies will need to recoup those development costs once it stops being about jockeying for position.
I don't even need today's frontier, give me a local model I can run on my Mac comparable to Claude 4.5 as of December last year and I'll probably lose any interest in new hosted LLM advancements altogether.
If you've been writing off Deepseek V4 Pro, now is your time to go set up moonbridge and give it a shake. It's exceptionally good.
Got a bit more than 1B tokens for $10, it's exceptionally fast, it was able to fix/implement things that 5.5 xhigh struggled with, without trying to act like my best friend or do that coy "undersell the ideal end result so that it can later overshoot it and claim a great success" bullshit.
E: miss me with the "but China" BS, everything I've experienced while using this model has convinced me they are earnestly more concerned with doing the right thing than Anthropic could ever pretend to be. And if you want to ask it questions about Mao, you can go download the weights and spend mid-five-figures to fine tune that out.
For US citizens: counting on Open Source AI is another libertarian fantasy.
Open source AI should and will get better for sure (including better defined first), but the state will have the power over AI never the less.
If you don't like govt's AI policy or the people making those policies, go fix that, don't act like you can avoid them.
For Chinese: saying "Open source AI must win" sounds like singing "L'Internationale, sera le genre humain". The reality is Open Source AI will be over the moment US competitive pressure gone.
For rest of world: there's no real AI for you so far, go work on it or be a citizen of US&A or China.
Civilization is at a crossroads, or will be soon. Democratization of AI can be good up to a point, but existential threats can also be real, and democratization of existential threats is not a survivable policy.
It's actually the opposite. Democratization of intelligence is the only way to stop existential threats and render them useless.
Right now, and likely forever, because biological threats can be sanctioned at a supply-chain level, the risk of AI is all digital. Fraud, phishing scams, spam, hacks, etc.
The only way we harden the worlds infrastructure to the point that it can withstand attack from bad AI is if we have an abundance of access to frontier intelligence to develop countermeasures.
Otherwise, bad actors will develop these capabilities behind closed doors and use them to hold the world hostage and cause irreparable harm. There's no putting the genie back in the bottle. Good and open-access AI and the people using it are the digital immune system.
If there's an asymmetry where bleeding edge is gated off to only a small group, and allowed to gain exponential power over the immune systems defense grid, the slightest infection will lead to death of the host.
Available components must win. I’ve often been a critic of open weights and open architectures that give very few normal people access. What’s the point of releasing the plans for a nuclear reactor if no one can have the fuel?
what if grok went open source and was on par with open chinese models? the business play may not be the models themselves but owning the data centers and running infrastructure for all models from all companies? a lot of people could then be rooting for xai and elon could ironically save face by actually implementing an open model
I hope so. But how? Who gonna fund these projects and how to coordinate with every sides. This is complex. I only believe that the open source AI won’t lack users.
Let's abstract this further: It's about the user's existing power and intentions, meaning if I am already in a position of power, AI will multiply it to levels way beyond a peasant could. Power dynamics just get exacerbated.
Winning is a tall order. I'm just hoping it'll get good enough while allowing us to run it locally with no idiotic "safety" controls or censorship of any sort. Looks like the best open weight models are at Sonnet level, if they get to Opus 4.6 level it's gonna be perfect.
Does he mean that the _best model_ should be an open source one (eg: today, something better than Fable 5), or just that open source models should be the default choice for most task?
The former seems an impossibility, closed labs can work off of open and their own closed research. Closed source will always be better. Well, at least until some late-stage enshittification dynamics cause the providers to hobble them.
The latter, becoming a default, not so much. But considering the deep-rooted nature of (for instance) Google, it certainly won't be a walk in the park. This seems to be a similar hurdle as dethroning Chrome as the default browser.
For the average ChatGPT user, I surmise that open-source models are already capable enough. Most people I know who use it (me included) are not paying for it, they are routed to the cheaper models.
What's needed here is everything else other than the model to be in place. Which is to say there isn't a sufficiently good open source ChatGPT app, every open source option requires more fiddling than the ChatGPT app.
No precedent comes to mind for non-tech-user software that is open source and also a default choice. The limitation is rarely from the core-tech capability; core-tech is often the same as what closed source uses.
Definitely, but I see the gap widening everyday, especially while commercial AI models have started converging towards AGI. However I do believe and support the cause, as it's the next big thing as developers we need to take to prevent a complete monopoly in the coming few years.
These things can't even center a div correctly half the time.
Not everything is code. Just because it generates a shitty SaaS clone for you and that seemed magical, it does not mean we are approaching "AGI".
An AGI could design an Oil tanker, manage the project from start to finish, handle all contract negotiations and purchasables, payroll, scheduling. Then it could do that 50x over and start a leading logistics firms.
In reality an LLM can't even complete upwork projects that are worth $20 an hour more than 4% or the time.
4% guys, 4%. It cannot complete entry level work on fucking Upwork 96% of the time. Stop falling for the marketing and sorry but an LLM will never be AGI.
Its literally just text autocomplete with some RLHF post training, holy shit im losing my mind. I want this hype to end so badly holy shit I need this to end.
It sucks how in just a few years the world has decided nothing is worth doing or is just impossible without the use of AI. As if regular human intelligence isn’t enough anymore and it has to be paid for somehow.
There is no such thing as "Open Source AI". Open Source means that you respect copyright. The types of AI models that this web site refers to do not. Stop this nonsense!
To me it does not matter whether AI is open source or not. Yes,
it is better if it is open source, no doubt, but either way I
think AI must die. Naturally it won't, we all know that, but
this does not change my statement in the slightest - AI must
die. Having it open source is, while an improvement, just
painting lipstick on the pig.
If you take AI risk seriously then Open Source AI should not and must not win. Both by evil actors (biological weapons research) and the danger of unaligned AGI itself. There are some people who would never work for the military or Anduril (automatic weapon systems), but an OS AI „without asking permission“ would be the same.
If closed-source AGI wins, it is not going to be much different from a safety perspective anyway, because AI capability research is advancing faster than safety research.
Closed Source AI at least can be controlled. See the directive of the US government regarding Fable (even if one disagrees about the directive there is no doubt that it is effective in shutting it off) or the safe guards by a corporate structure (even a profit driven one). It is schizophrenic to praise Anthropic for refusing the Department of War full access to their models but at the same time root for Open Source models.
> In terms of bioweapons, I expect that closed-source AIs will be heavily optimized against helping with these, and open-source AI will be banned after the first warning shot (or become economically prohibitive even before then).
Note that warning shot in that blog post means specifically a near-disaster event (perhaps one that's just barely averted) that's specifically caused by the AI. So far we've had AIUI no significant indication of open-weight AIs being problematic in that sense, whereas one can quibble that proprietary AIs have done dumb and dangerous things.
(For example, I suspect that plenty of folks would view the recently threatened mass scan of the DN42 hobby network as an instance of misaligned agentic behavior that would have wasted non-trivial resources, and I also think that most observers would pin the specific behavior of that AI on a proprietary SOTA model, not an open one. That's clearly not a disaster-level event, but it should scare you if you're concerned about alignment.)
This is not about information but about capital.
Even if we had free access to the weights of the best models in the world: who would be able to run them?
Technology is deflationary. I am holding in my hand a device that would have been a supercomputer 30 years ago. It costed me a couple of hundreds of dollars.
These models and the hardware they are running on will get even more efficient. We are nowhere near the physical limits of what we can achieve.
Not anymore! Well, if you're like Elon and already taking down the bottle of Cuatro Comas from the high shelf, the economies of scale will continue to work in your favor.
But one of the really neat things about AI is that there is no limit in sight to the scaling incentive. More compute will always get you more: more training, more inference, more parameters, more capacity to build more and better models, more spare capacity to run the slop your models have already built to generate the slop that will succeed it. Back in the dot-com days, or even the "big data" days, you wanted to scale up rapidly but there was a limit: there were only so many customers and they could only produce so much data you could only ingest so fast. In the late 90s, one of the world's most trafficked sites, ftp.cdrom.com, ran on a (single!) dual-processor Pentium Pro system. That was just serving files, and there was certainly room for more CPU oomph to provide more sophisticated services to a huge customer base. But once those customers were served, more compute, storage, and network capacity didn't buy you enough to justify the capex. That is emphatically not the case with AI, and so the incentives for the AI companies are to buy as much compute as they possibly can. What this means in practicing is pre-purchasing capacity at the semiconductor fabs to manufacture chips exclusively for you, and there's only so much of that capacity in the world. Trillion-dollar companies can easily outbid the entire consumer market, and so the incentives for the fabs are now to sell to AI companies at the expense of the consumer market. That's why you're seeing memory prices go through the roof. Modularized RAM for end-user PC builds will soon go the way of the CRT: it will cease to exist as a market product, it won't be manufactured anywhere by anyone. GPUs, CPUs, and storage will soon follow. The only devices end users will be permitted to purchase are all-in-one integrated devices, with CPU, RAM, GPU, storage, and networking either integrated in-chip or soldered on, and they will have just enough capacity to connect to the cloud services the user wants most to use. Most likely, you will be permitted a subscription to such a device, with automatic hardware upgrades at periodic intervals supplied by the manufacturer. If your subscription lapses the device bricks itself. Almost certainly, the OS will be locked down, with no end-user option to install a different one or even run unapproved software.
If reasonably powerful computer hardware for end users exists in this future, it will be available from a single company: Apple. Only they have the leverage to prevent ~100% of manufacturing capacity from going to high-roller, big-tech firms.
> Trillion-dollar companies can easily outbid the entire consumer market
I don't think this is true. I think prices are rising at the consumer and prosumer level because that's what's required for the mass market to collectively outbid the handful of trillion-dollar companies, at least for the limited share of production they can sustainably demand. This process can continue pretty much indefinitely.
> But one of the really neat things about AI is that there is no limit in sight to the scaling incentive.
How you can be so confident? I can imagine there is some limit and with each scaling iteration gain you achieved will decrease so that further iterations would be more and more look pointless
I'm sure a limit will come around eventually. But plans are afoot to build city-sized data centers, and even then that's not enough to sate the AI superscalers' ambitions, hence Elon's talk about putting data centers in space. This is a level of compute scaling unheard of in our lifetime, and we're still a long, long way off from AGI. So while the juice may theoretically not be worth the squeeze at some point, with the current capacity we have there is no end within sight to the incentive to build more. It will take a number of years at least, and who knows how much environmental/economic destruction, before the dropoff in return on capex begins in earnest.
Well it would be anyone that has access to a datacenter to run them. Which is a ton of companies. And those companies will rent out access to those models. And if they do something stupid to screw over consumers, well the whole point is that there would be a bunch of companies that you could use instead.
Anthropic just kneecapped themselves, and possibly OpenAI and Google as well, with their FUD strategy that got fable shutdown by the government.
But that doesn't impact Chinese providers. Then can US companies get investments for expensive model development if they can't actually sell those models-as-a-service?
In the meantime, open source will continue its march onward because while slower, it's completely open source, and the models are already good enough to improve their own work as well as build out the next gen of models.
So I've long said that the valuation of OpenAI at a trillion(ish) dollars depends on OpenAI "winning" and "owning" AI and there being a sufficient moat to stay ahead of competition. Without that, the company is worth a fraction of that. Anthropic is probably positioned better here actually but it's still kinda true there too.
Ever since a Chinese firm released DeepSeek I immediately came to the realization that any US tech firm "owning" AI is simply not going to happen. China will make sure of it. It's in their national security interest not to let that happen.
From the POV of geopolitics, IMHO the US shot itself in the foot by banning the export of the best chips to China. The US also somehow has the power to prevent a Dutch company (ASML) from selling to China too. That makes a little more sense to ban but the combination of banning EUV exports AND banning the best chips sowed the seeds for the destruction of all of this.
By banning chip sales, the US inadvertently created a captive market for Chinese chips with Chinese companies. If there were no chip ban, Chinese companies probably would've bought US chips. But they can't. So they can only buy from Huawei and SMEE (indirectly). The US forced China to realize it was in their national security interest to copy the best lithography and, by extension, the best AI chips.
So DeepSeek was reportedly developed on either older NVidia hardware or smuggled newer NVidia hardware but that won't last either. At some point it'll be completely Chinese made chips that are doing this.
And what's the biggest cost for a model? Training. But you do that once and the model like any software is infinitely copyable so China can under OpenAI, Anthropic and SpaceX (xAI) and that's what they're doing.
But it gets worse for the AI moat. Local models are going to get cheaper and cheaper to run. You can already run 31B models on sub-$5000 hardware. What do you think it'll cost in 5 years? Will larager parameter models keep getting better or will there be a law of diminishing returns? What is a B100 workload now, will be a Macbook Pro workload in as little as 5 years.
What if all these AI data centers are ultimately just going to be commoditized cloud hardware like AWS in the not too distant future? We already see Google renting big from SpaceX. I think the writedown on all these data center investments and the companies that are doing them is going to be extreme in the next 5 years.
I hope the news moves this debate past "open weights vs. closed APIs" as the only axis. Open weights matter, definitely, but applied AI also needs open infrastructure around the model and it feels a bit like I'm yelling into the abyss highlighting the future we're incentivizing - cognition rented from a few institutions with access changing based on policy, geopolitics and platform incentives like advertising
I mean, even if the frontier labs opened their frontier models, only nation-state level actors are capable of running them. A lot of the tech is very open and known, its putting it all together that's the struggle.
If any AI wins, how can that be good for humans? It's high minded but if any AI wins, why would any of "The ability to study, build, repair, deploy, audit, adapt, teach, preserve" be important? Is the real problem to be solved something else, if you want those things?
Hear me out, economies of scale can only be met when there is a large enough liquidity for it.
The amount of people willing to purchase multiple hardware releases year after year just to run LLM is already tiny and businesses already do use their own hardware and there is no desire for manufacturer to reduce their own margins.
it is baffling that you can still encounter Yuddite delulu in 2026 when everyone and their literal grandma is using chatbots daily. you might as well campaign to shut down the internet or ban smartphones.
but ok, who is going to initiate such a treaty? US? the orange man won't, and even if he did, no one would care. by the time his term is over and the next AIPAC spokesperson is elected, it will be even more late than it is now. EU? impotent and irrelevant. China? lmao.
Given that it's most public use in open source so far is to whitewash GPL code into MIT code, no, I'm sorry, I don't think "open source AI" is particularly important.
But if "they" stay on the current trajectory we'll never own hardware capable enough to run the open source AI. They want us to rent everything from the cloud and never own it. If a government-supported cartel forms around this idea (which appears to be the case) that's the end of it.
Open source projects are only successful when they make what they replace obsolete. This worked with Linux and GCC but this isn't gonna work with LLM's.
Who's gonna pay to power an open source AI? Will it perform well enough to make Chat-GPT and Claude obsolete?
Not to be that guy, but the correct term is Open Weight LLM. And I’d argue it already has. Many open models are already very competitive with closed models at a fraction of the cost.
Were it not for China, America would have restricted the most advanced models from being used outside the US. NATO members would have access to GPT-4, with some countries entirely blocked from AI.
Biden's GPU controls should give you an idea. Thank you, China. Open source AI must win.
China unironically saved humanity. I'm no fan of the CCP but if they hadn't organized an effort to compete with the US no one else would have done it and we'd be begging our AI overlords for tokens and praying we don't get caught conducting wrongthink.
Go ask Claude to criticize Anthropic and see how long your account stays active.
> If intelligence becomes something people can only rent from a few closed institutions, the public does not just lose software freedom. It loses operational freedom.
And people do not just lose operational freedom. They lose the freedom to think, much less act. To some extent, general intelligence has already been outsourced to a few companies. Phones and computers extend the human mind's capabilities, but most people don't have root on their phone. They don't know or control what software is running on it, or how the hardware is made. They don't control their phone, the phone controls them instead. The upstream problem is ownership of general computation, ownership of your own mind, aka self-ownership. This will become more obvious as computing devices become more personally integrated (desktop -> laptop -> smartphone -> smartglasses -> neural interface). Who owns the digital part of your mind? It's not really you at the moment.
Democracy, or any form of negotiation, can only exist among entities with similar capabilities. The gap must be very small. Orangutans may be smart enough to drive a golf cart, but there are no orangutan citizens in a human democracy. So you cannot run from this by being a luddite hermit in the mountains. When the world is full of digitally computing humans much smarter than you, you'll be at their mercy like monkeys are at the mercy of humans. We destroy their habitats and experiment on them as we please.
Now for the first time in history, organisms can increase their own information processing capability at will. We're in the middle of a speciation event where humanity splits into those who own the digital part of their mind vs those who don't, and there will be further splits based on how much compute you own. Though a future where no individual can fully own their mind is also possible.
By "own", I mean being able to command the entire technology stack. If we want sovereignty for the masses, then we must decentralize the entire technology stack for general computation. That means everything from electricity generation, to chip design and fabbing, to all layers of software from firmware to neural networks. All of it must be accessible to every individual. Everyone must be able to make a computer from scratch at home, or at least without leaving the city they live in. Anything less than that, and democratic society as we know it will continue to crumble.
The fundamental idea underlying all of this is: that which reproduces, survives.
At what level of organization can we reproduce?
The digitally computing human species cannot reproduce as individuals. We can only reproduce as a society, at least for now. You can't make a computer from scratch on your own, but you can make a brain from scratch with just one other person of the opposite sex. As the world we live in becomes more suitable for the digitally computing rather than the purely organic, the organic part of the digitally computing human becomes less likely to voluntarily reproduce. If the organic part were to survive without being disempowered in the future, then it's probably by moving the mechanisms for reproductive drive to the society level (via religion or authoritarian government incentivizing or mandating reproduction), or by ensuring that each and every individual has the means to make the digital part of their mind on their own just like how they can make the biological part on their own.
A website stating the obvious, given small target audience it will probably reach, and a call to arms consisting in emailing a random unknown person.
We're saved /s
Instead of doing a vanity site with a shelf-life of a few days, see where the action already is in online local LLM research and communities and contribute.
On this very thread you already have people talking about "open weights" and similar nonsense. What is open about them? They're free to download, but that hardly qualifies as open. Where is the source? Where are the instructions to modify and build your own?
I'd never though I'd have to utter the expression "open as in beer".
The blatant attempt at manipulating vocabulary here is... quite blatant.
I'm a strong proponent of Open Source (TM) but I disagree with this take.
The weights are the useful artifact here. You can modify them, fine tune them and do what you want with them.
Unlike binary software there is nothing limiting that.
It is also useful to have access to the training recipes and to some extent the data. But I'm of the opinion that learning on something is not copyright infringement, so there are many circumstances where distributing the raw training data will not be possible.
For me this is like Open Office: it is open source, and largely inspired by and learned from Microsoft Office. But they don't need to distribute MS Office for Open Office to be Open Source.
In addition there are models that meet the criteria you appear to propose. The AllenAI models are a good example.
The analogy falls apart very quickly. Without the training data, your modifications amount to virtually nothing compared to what these "versions" are, and the idea that you can maintain and improve on these models without the continual support of the company that owns the training data AND harnesses AND in general build instructions is not very credible. This is why it's not rare that they "dump" old versions as freeware but at some point switch to not distributing them, and mostly get away with it. As this is really not open, and the threat of an effective fork is therefore non-existent, the pressure for any one who has released freeware models to "go SaaS" is too high.
While if "Open Office" switches to a more problematic license at some point, the existing source has all you need for an organization to support the project without regard to the original company (this has happened already!). If Qwen decides to stop distributing models for download, you're basically stuck, _even_ if you have unlimited resources, it's not clear how the released weights help you; your best bet is to start almost from scratch. This has also happened...
These models are not "Open" by any definition of the word. It is just freely redistributable. You can justify yourself in whatever way you want re a cowboy approach to copyright, but this doesn't change the fact that this is not open, and has almost none of the benefits of open, and therefore it is a huge abuse of the word "Open".
Ironically about the only thing that is copyrightable here is the sum of the training data (possibly) _AND_ the software used to build the model (most definitely). The model itself most likely isn't (databases are not copyrightable), which makes it even more pointless to abuse the word "open" for it. All the value is in the former two.
This, and distributed LLM inference. We are at a point where no single person can setup a rig to run a SOTA model, it is just too expensive.
So we must build and adopt frameworks that allow individuals to share resources to run SOTA models in a distributed manner. That way they will also be non-censorable by governments.
Also The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it.
I propose the TokenTorrent protocol. Hoist yer pirate, erhm, clanker flags!
I wonder if there is way local small LLMs can complement each other in away that the sum-total yields a much more performant LLM
Perhaps some radical MoE where you download _exactly_ the components you need as you need them. Currently MoE is switched usually on per-token per-layer basis, so you need all weights locally. But e.g. Apple made one which pre-selects all experts based on prompt embedding. That might be further scaled up - e.g. predict exactly what's needed
> The only way to prevent that one entity weaponizes it, is by giving EVERYONE access to it
There is a middle way; the policy space also includes government regulating both access and monopoly.
I’m opposed to monopolies of this tech, but I hope the risks of giving everyone jailbroken AGI/ASI are clear.
As a toy example you could imagine a Universal Basic AI where government subcontracts to (n_quorum) labs, everyone gets a token budget, but operating the APIs comes with the safety controls.
If everyone does get to run their own jailbroken AGI, then the only stable societal norm I see is A LOT of surveillance to make sure nobody is building CBRNE threats. This doesn’t seem like a clear win from a civil liberty perspective, though I could see the argument.
We have nothing anywhere near AGI/ASI so you're good for another 25 years, my friend
That is exactly what ASI wants you to think. foil hat off
yes, it also complements the geohot idea behind the tinybox
What is that? I can’t seem to figure out what the use case is vs buying off the shelf?
I think it’s a great project but the communication isn’t clear to me.
https://tinygrad.org/#tinybox
I'm not sure exactly why you would buy through them vs rolling your own if you could afford the equivalent hardware.
I'm a firm supporter of local inference though so good on them for doing something
Lol I get nervous when I see a list of products with full specs but no prices
They have prices. Click on the links in the shipping row.
And..... I was right to be nervous
I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable.
And, there is the issue of data poisoning from untrusted nodes. I've almost cracked that last issue with a self-healing checkpointed rollback system that doesn't have to throw out anything that follows the corrupt datum.
But, I'm just one person with an idea and I don't have infinite funds to make this happen. This isn't a small project.
Maybe there would be interest in something like this, now that entire frontier labs are being banned from making further progress.
The total power of all GPUs on the planet dwarf their capabilities, if we had a way to harness them in a distributed way efficiently. We wouldn't be able to train a Fable as fast as them, but eventually having access is better than never having access.
As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea.
The far, FAR superior power efficiency means that even if you did harness every public GPU or GPU-like device on earth, you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.
And even if electricity was free, having those GPUs spread over the world with internet-level latency will slow everything down by factors of thousands to millions - if it's feasible at all. Regardless, you're not getting fable-oss this decade, maybe even not this century.
It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do.
AI hardware is for inference, not training. Training uses normal HPC crap. Superpods aren't really power efficient, it's kind of a meme, and it stems from limiting the power draw of other components by having less of them. It's more of a rounding error.
> you'd end up consuming so much excess electricity it would be cheaper on net to simply take the money that would have gone to the power bill and spend it on your own datacenter.
Costs spread over a large population, it really doesn't matter. You're not getting hundreds of thousands of people to pitch half their monthly electric bill to pay for someone else's datacenter. They will pay the electricity themselves quite happily though, if all they need to do is give you compute. This isn't new.
Interconnect is the bottleneck for distributed training, nothing else really.
You got it wrong. Inference can use crap GPU's. Training needs the 100x more expensive big guns. Our training machine is 100x more expensive than our inference machine.
How is the result of training stored? How big is that? It seems reasonable to assume we’ll eventually plateau and all we’ll need is relatively infrequent training.
I agree, leave the training to open source federations that roll out like operating systems. Minimal training over time.
Then have inference go down to the next layer to use those models as a P2P decentralized network.
Maybe like open router could tap federation networks.
> AI hardware is for inference, not training
Not sure what you are referring to, unless you don't think h100/h200/b200 are "AI hardware"
> Superpods aren't really power efficient
Maybe not compared to a specialized rig with multiple 4090s, but that is the best case for consumer hardware - the vast majority will be dramatically less efficient than that
Anyway, I agree the interconnect is by far the biggest obstacle and seems insurmountable, I should probably have led with that.
Bit of a doozie though, that one.
I recall getting really excited over hinton's FF foray, right before he bailed on AI as a societal direction (which, if anyone ever had the right, I suppose he does). If one squints, one can see a backprop-free base being much easier to train on geographically distributed and heterogenous hardware.
Are you sure most of frontier cost isn't inference in RL environments?
That makes no sense. It’s basically the same calculations for training as well.
Dunno, in a sense, torrents came among similar restrictions. Everything at consumer level was just plain awful and at dial up level, mebbe ISDN if you were very lucky, with fiber only available to ridiculously rich people and corps. But with restrictions, came approaches on how to mitigate them.
Yes but not violations of the laws of physics. You need extremely fast communications, memory bandwidth, etc; you cannot get that with distributed training. You're up against the speed of light and the interconnect that powers the internet. You will always have horrifically slow latency compared to if you pack the servers together in the same place with specialized networking.
If weights can't be looked at almost instantly in bulk, it just doesn't work. It's a different problem from distributing file downloads.
Could you put some numbers and examples behind the efficiency gap between data center and consumer-grade AI hardware? Did you include examples like the RTX Spark on the consumer side? I was always amazed at the low power consumption of unified memory style architectures. In absolute terms and even more so compared to consumer-grade GPUs. I'd be genuinely interested in a comparison with data-center-grade hardware.
It's more than the raw hardware, it's the interconnect and communication between the hardware at scale. These models are trained on hundreds of thousands of GPUs today. You _will_ start to see cross-datacenter training runs but this needs to efficiently decide when and how to communicate across datacenter, which bears a very high cost compared to intra-datacenter communication.
DGX Spark is effectively prosumer hardware, better than most consumer stuff but still not comparable to actual datacenter gear. You can't just look at TDP in isolation without also comparing performance.
> It would be better for governments to buy and own their own datacenters, maybe as a coalition, and dedicate their operation to the public good. I believe that is what we actually have to do.
100% agree. The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization/global realignment is going to push us into a death spiral, and even if AI is a smash hit, the ~19% federal capture of corporate revenue isn't nearly enough to pull us out of it. The people owning the compute infrastructure and capturing more profit from AI at that layer is the safest, cleanest way to increase revenue capture, a sovereign wealth fund is a mediocre idea because it's possible to play shell game with stocks and redirect profit/debt (venture capital is quite good at this!).
> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it in order to fix the economy, as the combination of debt burden and interest rate pressure from de-dollarization/global realignment is going to push us into a death spiral, and even if AI is a smash hit, the ~19% federal capture of corporate revenue isn't nearly enough to pull us out of it.
Any actual numbers to back this up? I don't see how nationalizing a very cutting edge technology outside of wartime is going to go super well. The leverage that these companies have is the same leverage that TSMC has: you can't just take over and expect things to rocket at the pace its going
>> The US government basically has to nationalize AI and capture an outsize portion of the revenue from it
Currently AI has generated no profit. And as it sits, is a non viable business.
I refuse to include the sellers of shovels as AI revenue.
If the companies buying the shovels are still losing money, then the tool supplier fortunes have nothing to do with the economics of the AI application layer, who is losing money on every prompt.
It's the most naive opinion that keeps getting shoveled around. You have a product that is viewed as essential by businesses, with revenue growing by 10x a year and geopolitical ramifications that have continued to rear their heads and your opinion is "this is all an unprofitable shill". It is extraordinary to me that people really believe this. Whether or not labs run at a loss today is absolutely irrelevant. There is of course steady state economics that make sense, and its currently not well known what the profitability picture is right now, so to say "Currently AI has generated no profit" is also just speculation and not a very insightful one at that.
I've heard that the API calls by themselves are ~60% profit if you ignore capital expenditures. The labs haven't generated profit because they're constantly sinking money into the next generation of larger models to stay relevant. Dario has talked about the economics of this a lot, and I do believe him there.
There's clearly also a lot of pent up demand in the corporate world for inference, the problem is that it's currently expensive enough that enterprises are balking at the cost before they've had a chance to refine processes and see projects through to fruition. That's a tractable problem to solve though.
The number of capital-heavy businesses that are wildly profitable “if you ignore capital expenses” is too many to list.
Airlines, for example, which are so profitable they continually go bankrupt.
That's true, but if the frontier doesn't advance there's no depreciation or ongoing capital expenditure. If all the frontier labs agreed to stop making stronger AI and just try to sell what they've already trained today, their books would turn green in a hurry.
Efficiency difference between training on GPUs and TPUs is 2x at best. You can get very efficient with tensorcores, converging to TPU efficiency. In the end math is math, you can't make a multiplication more efficient than it already is on GPU.
I guess this was more related to syncing GPUs.
If you were to take 500 computers with older 1080 GPUs, you might have enough compute/ram equivalent to an H200 GPU for training such a model. Maybe take 10000.
But if those machines are spread over 10000 homes, wired with residential internet service, training a large model will not get anywhere.
You go from "data in the same HBM memory chip" at 4.8TB/s or "data in adjacent GPU" with NVlink at 1.2 TB/s down to 25 MBit/s upload speed. Accessing the next piece of data is going to be about a Million times slower. At the same time you will heat a thousand times more, for a Million times longer.
You need to train independently and merge rarely. The problem is the merge step. Weights are too entangled, you are not going to get an improvement commensurate to the effort. Otherwise, everyone would do it. It is an open research problem.
That sounds like the way. Everyone trains their own small problems to maximally compressed weights and then merges.
The power-constrained part of compute is data movement, not the elementary arithmetic per se. Anyway, it's very possible to tweak the underlying design to increase throughput a lot for any given power budget at the cost of high latency. This seems especially useful for training workloads where we don't really care about latency as much.
Math is math, but sadly math isn't physics nor engineering.
math has physics.
WRT government data centers, there is certainly precedent for independent researchers getting HPC time on systems owned by US national labs, research institutions, universities, and then publishing their results as part of the public good.
One would question why this hasn't already happened as the rule and as opposed to the proliferation of private data centers. However, I am sure the answers are plain and perhaps saddening to us all.
> It would be better for governments to buy and own their own datacenters,
I mean thats good, but they'd have to also build thier own dataset. Which involves either paying people, or breaking the law.
Plus if they do manage to make it work, they will not get any tax revenue from it, as it'll remove the need for labour, which is where a huge amount of tax revenues come from.
its a deeply hard problem with lots of second/third order effects.
What makes you think Deepseek or GLM won't catch up to Fable level? Why would there be a break in the trend now?
DeepSeek and GLM (plus Kimi) are at or above Sonnet level wrt. favorable workloads like coding. They're not close to Opus or the latest GPT yet, and Fable is even higher than that. Other workloads relying more on real-world knowledge have them even further behind, and this can't be mitigated without making the model itself bigger and harder to host locally.
Not true. Big models buy you baked in knowledge and long context cohesion. A model can be trained to use search and knowledge base tools more efficiently to mitigate the former, and harnesses/workflows can be designed to push models into small parallel threads to mitigate the latter.
The thing that big models will always bring to the table is the ability to YOLO weak/under-specified prompts, and spend less time in the loop making sure work gets partitioned correctly. For smaller/simpler tasks the P(success) difference isn't that big.
Knowledge-base access is not very useful in general because a model doesn't have well-defined "known unknowns" that might trigger an agentic search of the outside knowledge base. Plus surfacing knowledge you don't know much about is itself hard.
These things sound plausible, but have they actually been demonstrated? Wouldn't anyone who succeeded in making such a small but useful LLM be raking in the money now?
Cursor's composer 2.5 is a perfect example. It's right on the heels of the frontier (for coding only) for an order of magnitude cheaper. As much as I've shit on Cursor in the past, I do think the company is well positioned to pick up people getting sticker shock on Anthropic tokens, if they can get their marketing down.
If that's Kimi-based it would very much be on the larger side of open-weight models (1T params).
It is, but the US labs have been pushing parameters heavily. There was a pullback from big models after GPT4.5 in particular, but with a shift towards emphasis on post training and the good results Google got with scaling Gemini 3, all the labs started to push scaling again, which is the reason the frontier is getting more expensive. So that 1T isn't as big as it sounds, the American frontier is probably sitting at 3-5T at least.
> They're not close to Opus or the latest GPT yet
Disagreed. GLM-5.1 is easily as good as Opus 4.5 for all the coding purposes I could throw at it, which is the model that kicked this entire hype cycle into overdrive in the first place.
I've found GLM to be comparable or better than Opus at writing and at a fraction of the cost.
Writing does not rely on real-world knowledge all that much, other than knowledge of language itself. Even tiny models can achieve that, it's even easier than coding.
The challenge with writing is the lab collapsing the distribution around "tasteful" writing, when the people making decisions about training data aren't able to effectively discriminate it.
The key thing here is that effective intelligence = model capability / cost. If you drive down the cost of inference you can have higher effective capability even with a technically less capable model. There is nothing in Anthropic/OpenAIs general reasoning capabilities that can't be easily done much better with a purpose built harness for a domain specific task.
I think there are at least few question marks.
One being that extrapolating from like 3 data points is hardly science. All trends break at some point.
The other is that the measures to prevent distillation of their models (if it was a secret sauce of Chinese models) could work if nobody is allowed to use them.
> As I replied to a child comment - this is a nice idea that just isn't tenable in reality. AI hardware isn't just hilariously faster than consumer GPUs, it's also hilariously more power-efficient and has hilariously better connectivity. Every one of these dimensions kills the idea.
The first part is not really true though, the chips are not that much faster, the DRAM is not that much faster, and in aggregate it does not matter because there is just so much more consumer hardware out there (although perhaps that is changing as supply shifts toward datacenters).
The interconnect and data locality is the problem. If you could train it like e.g. you can render a scene with monte carlo ray tracing, any result from any node could be merged with any other and the combined result would have converged closer to the limit. I am sure research in that direction exists, it just has not proven effective within the scales it has been attempted.
>But when people think of decentralized training, they don’t first think of gigantic datacenters, owned by the same company, training models across large distances. Instead, they imagine thousands of small datacenters, or individual consumers, pooling their spare compute over the internet to orchestrate a training run larger than any single actor could manage alone. Many companies are pursuing this vision: Pluralis Research, Prime Intellect and Nous Research have already successfully decentrally trained models at scale. But in practice, training decentrally over the internet has lagged far behind more centralized training. Even their largest models (Pluralis’ 8B Protocol Model, Prime Intellect’s INTELLECT-1, and Nous’ Consilience 40B) have been trained with 1,000x less compute than today’s frontier models (such as xAI’s Grok 4). https://epoch.ai/gradient-updates/how-far-can-decentralized-...
I think it's fundamentally not useful as long as there are other open source model releases. E.g. suppose you make SotA model at a particular size via decentralized training. Amazing. In a month Qwen/Deepseek/etc release a new model which is better. So why would you use the "decentralized one"?
Models have limited shelf live while things are improving rapidly, and decentralized training is just more wasteful.
However, things might change if we get to what Karpathy calls "cognitive core" - a stable model backbone which can be extended via skills/adapters/etc. Development of extensions to the core can be a lot more decentralized.
But for now these decentralized training attempts function largely as a deterrent to anti-open-source collusion
> The total power of all GPUs on the planet dwarf their capabilities
That just isn't true. It misunderstands exactly how much silicon has gone directly to those companies, and exactly how much more powerful said silicon is compared to consumer grade gear.
If folding@home is a useful yardstick by which we might estimate the amount of GPU-ish capability that civilians might be coaxed into donating to a shared enterprise, yeah, it doesn't look pretty. This is extremely rough napkin math but comparing to xAI's Collosus 2 for example, for training workflows you're probably looking at 4-5 orders of magnitude the capability of all of folding@home combined. That's 100,000 times faster.
Very rough math like I said but I doubt it's directionally wrong.
And even if you did force literally everyone on earth with some sort of GPU to max it out 24/7 in service of an open source AI training enterprise - you would waste so much power trying to use that inefficient consumer hardware with the worst latency imaginable that it would be cheaper and faster to get everyone to instead chip in some cash to buy a datacenter with blackwell chips instead! So the idea has no legs whatsoever.
Plus a scientific project to benefit all of humanity doesn’t have quite the same ring as the thing thats stealing your job, from the volunteer’s perspective
folding@home reached 2.43 exaflops by April 12, 2020, which would make it the largest supercomputer on the planet.
it's down 99% since that peak. But let's compare to it anyway.
It's pretty useless to compare raw FLOPS, but as a general hand-waving guesstimate, F@H is currently doing about 25 petaflops in a mix of FP16 and 32. AI usually trains at FP8, but to keep things fair the H100 is quoted at 60 FP64 teraflops per unit, so that's 12 FP64 exaflops given its 200k count.
So F@H at its peak did 2.43 exaflops@FP16/32. Colossus 1 does 12@FP64. These numbers are very hand-wavy, but I think the point is made.
By the way, I'm not trying to crap on F@H - I think it's an outstanding project and I've run it in the past. But a volunteer group simply cannot compete with well-funded, concentrated effort like what's going into AI.
there was a project trying to achieve some of those goals a few years ago using p2p: petals https://github.com/bigscience-workshop/petals
their bloom model was also a collaborative effort https://huggingface.co/docs/transformers/en/model_doc/bloom
I was wondering what happened to this
Maybe the training approaches taken to date are wrong for decentralized systems. Setup a virtual subnet you can trust and do training on that. Create a AI model island in a trusted/federated model system -- definitely slower than the typical 'one big model' approach, but scalable to world size modeling.
Also, it wouldn't be able to use a transformer architecture. For inspiration, take a look at Google Maps and how it a much more efficient A* divide/conquer hill-climbing architecture. Think minimized matrix math.
Other comments also hint at this idea, a distributed training solution is currently an open research problem. Solving it is not easy, yet. But 10 years ago what we have today for LLMs would have looked similarly impossible, so have hope, and apply yourself to the problem if you find it interesting!
The gradient info can be compressed 10000x with the right tricks, I think it is achievable. Nous claims they did it already:
https://github.com/NousResearch/DisTrO
There are other gradient compression papers from the past reporting large compression rates
Have you checked out [petals](https://petals.dev/) It’s doing the same thing, however the project is written in python and there can be some optimizations to make it much more faster.
Is the total compute capacity outside of meta, google, amazon, anthropic, oai and x is higher than even the capacity of any of them? In any case, there's no chance a public collaboration gets to anthropic levels of compute even if communication were no issue.
Is the issue that training with less compute takes more time? Or is it just not possible? I think a collective using distributed training could tolerate the idea that it takes 10x as long as Anthropic to train a model, or whatever.
It's possible but it's not linear. A modern AI training cluster is a supercomputer that uses very different architectures and hardware to a bunch of small PCs connected via normal networking. The networking advantage alone kills any chance of decentralized training.
This could be of interest to you: https://thealliance.ai/projects/tapestry
Man, that project is such bait for my particular sensibilities but just looking at the copy about not sharing your data and only sharing weights has me feeling very disappointed in the project already. I would want a project like this to not elide fact that sharing your weight updates probably effectively means sharing your data too.
there are some strong open source groups like NOUS research taking the fight https://nousresearch.com/
It seems this project is serious and very promising. They have the Psyche network which seems real and operational. They're able to produce ~50B-class models, this will only grow over time of course. Very cool.
>The communication speeds are untenable.
Can it be parallelized or not?
If you take a model, make two copies, and fine-tune each one on different data, what happens when you merge them? Does it work if you freeze different layers?
I think this works if the steps are small enough. And the transfer should become tenable if the steps are big enough. Where's the cutoff?
Yes it can be parallelized, it already is in real AI datacenters and no it doesn't help you. Like everyone else is saying, an AI datacenter is not just a bunch of gaming GPUs connected via normal ethernet and hasn't been for years.
At most a decentralized effort could contribute a little bit to some bigger centralized effort by doing inference and sandboxed CPU work. Modern model training isn't just backprop, it's got a huge and growing CPU and inferencing component too, which doesn't require intense inter-node communication. For instance, doing RL rollouts for agentic coding requires a lot of plain old inferencing and sandboxed containers for the models to practice in. The final results are just a set of rollouts and scores that can be uploaded back to a central datacenter for GRPO to adjust the weights (relatively cheap). But then, of course, you'd have to stick to models small enough to fit on people's computers so it'd never be competitive.
Kinda sounds like we just need better computers.
https://learning-at-home.github.io/
AI with blockchain. Maybe we can mix in IoT and VR for the ultimate in buzzword synergy.
Ya that'd be an awesome project, the only issue is how do you verify it's not being poisoned? To actually validate it would require more analysis than the training took to run. It would require a trusted network, not an open one, unless that can get solved somehow.
Make multiple nodes do the same job, compare results.
Could it be done by making a sparse MoE of thousands, or tens of thousands, of smaller experts in very niche domains? Maybe a tree-like structure of experts which can delegate from relatively general but inaccurate to extremely niche but accurate? Also these experts might be plug-and-play, easily swap out an inferior expert with a stronger one in the future without having to redo the whole pile?
That's not really how the experts in an MoE work. They activate on token probabilities and are activated on every token. You don't necessarily have a discrete math expert and a discrete physics expert. And if it were you would still need a router that is trained on all of those domains.
MoE models are typically designed for datacenter deployment, where per-token load-balancing is more important, but it's also possible to use a different training objective that encourages domain-specialization of experts: https://allenai.org/blog/emo But yes, this isn't really useful for distributed training as such because of the router.
There are some attempts at this problem, like Bittensor, Akash Network etc
Don't know but could BOINC setup which has been around for ages and mature plus has some incentive mechanism (Gridcoin) be used for this?
Well, I suppose it is understandable why you want to attack the most obvious problem with such a scheme: obtaining sufficient compute.
That does mean you are actually neglecting the more difficult issues.
The biggest problem is accuracy and integrity of the actors in the project.
Sounds like SETI@home but for AGI... SAGI@home?
Since SAGI can't be practically distributed, and it puts so many people out of work, how about moving all of the unhoused people into the nice warm data centers, and call it home@SAGI.
Or is that too close to the plot of The Matrix?
I believe we are not the only ones
we will be better of doing political activism for govt to provide open researchers and builders access to gpu in govt built dataceter
>I've been contemplating a decentralized model training system for some time using volunteer machines that we all contribute. But, it is astronomically difficult. The communication speeds are untenable.
It is already possible: https://arxiv.org/abs/2603.08163 . You don't need to sync so frequently, so it can be done over normal internet, it's just less efficient (takes longer to converge).
It won in my house/my business right from the start. (Well, open weights, at least — which is an uncomfortable nuance.)
I have never understood the willingness to make the functioning of or development of a product so completely dependent on the secret sauce of one of two big unprofitable, inscrutable startups.
It really defies sensible engineering principles to do that. So I was never going to do it. I'm exploring AI now but because I have decided that open weights make it a good use of my time.
It's bad enough that any given business often ends up beholden to a single payment platform and the policies of two US credit card providers.
I guess it is the freelancer in me but I always feel nervous when I am asked to put so much energy into studying or learning someone's product, rather than the underlying technology. I still remember the days when Microsoft was pretty much lobbying academic departments with promises of access to the NT source code. I remember a senior figure in our own saying that Linux was a sideshow and access to NT would make us relevant.
More control over destiny is always necessary, and I remind myself and others that the "state of the art" is behind the "cutting edge". Progress is made at the cutting edge, but there is risk of damage. Engineering should focus on building on the state of the art, not on hitching a ride on someone else's progress.
I feel like "open source" in this context is, as you say, an uncomfortable nuance; the tooling (llama.cpp, et al) is open but useless without weights.
The weights are extraordinarily expensive "capital" that is donated by big organizations who are all at war with each other.
I don't know that it will ever be possible for, for instance, archive.org, to make truly open weights. And, other than archive.org, I can't imagine any other "open source" organization (freebsd? apache?) being in any position at all to make truly open weights.
Maybe governments, government organizations, or universities.
None of whom are currently funded, mandated, inclined, or particularly interested in dumping the money into buying the infrastructure needed to make weights.
Yes. The weights war is a much more aggressive war than the war of OSS donations.
In the OSS donations war (Visual Studio Code being a really fascinating example of it) you could see that the taps can't be turned off so easily. Whatever is donated can be built upon forever.
I think there will come a point, soon enough, where open weights models are capable enough that even if they stagnate, they can be augmented with tooling that essentially keeps them current. Maybe we are there now?
But the risk of the taps being turned off is not negligible.
My own feeling is that governments will ultimately ask consortia of universities to train open weights models and support them financially in doing so.
(And for what it is worth, I think diffusion text models are likely to trigger a hardware arms race that makes this possible)
In much the same way that they used to do that for the supercomputer race, which we just don't hear about right now!
Interestingly, I've taken a different approach. AI supplements how my business builds and I'd much rather have all my engineers using Opus 4.8 rather than whatever the best open source models are.
I believe open source is important, but for my business I'm just going to use the best tools I have available to me.
As a business decision it makes sense if you think that spilling out agent-written code to meet some profitable objective is a race you can win?
I know I can't win that race or outspend the competition. So I have to rely on my instinct that in my area of business, people becoming dependent on agent-written code are getting further and further out of their depth, and that slow and steady will win the race. I am going to spend the time trying to integrate the open source tools into the way I work. (I am still working on this; frankly I may have bigger problems on an individual level than they can solve)
To be maximally clear, if this two-inscrutable-megacorps model does survive, and it becomes how everyone works over even the medium term, I'll have to quit tech.
I will probably retire early and just plan for a shorter, quieter life that ends when I am out of money, because like everyone else I won't be able to afford a longer one.
I don't want that "nobody prompts now, we just specify loops" bullshit for myself and I don't want what it will do to me for anyone I love.
Open source and open weights have to win for human culture's sake but in the short term for the sake of the culture of tech work. We need control over how we use these tools, not just to be steered down whichever channel makes the most money for Dario and Sam.
When "open source" means freeware, it's like saying "we want free copies".
What we should be saying is: We want a public, community-ran project that does pretraining and training collectively. This means working on a training corpus in public and somehow coordinating the training work.
This is a complete change of what the term means, It's like how people conflate piracy with theft. Two different things, use different words. Free weights, inference code and chat template is very different from a community-ran LLM project.
I would be totally willing to pay $50 per month to support an open source AI lab, rather to get open source models as byproducts of corporations.
Well, AI labs are spending hundreds of billions, so you're gonna need a lot of subscribers to compete...
Good point. I currently choose to give money to Anthrophic rather than OpenAI because they align a tiny bit more with my values and the product is good. Perhaps releasing an open source model every year could be a differentiator from competitors, where enterprise and individuals choses the lab not because is the best model out there but because gives autonomy in case something happens to the organization providing the models.
For now. Progress in hardware/model efficiency is one of the threats the big AI labs face, because if LLMs become commoditized they can’t make back the billions they spent.
I think OpenAI ruined the trust. How do you know this 'open source AI lab' won't somehow spin off a for-profit?
Because they are supported by the Chinese government for the purpose of undermining US capital investments.
The new cold war is weird.
Coups like that can happen due to organizations having a small number of board members that can decide to do as they like.
Proper mass-membership organizations are possible though. Same rules as a public corporation, but one vote per members, and the yearly meeting decides the board members and approves important decisions or introduce motions that steer the organization.
So the right way to do this would be to create something like the "Public LLM development club", some criteria on membership (after all entryism is a thing), some membership fee sufficient that there is money for a reasonable amount of work to be done and then one has to hope that people join.
So elections guarantee fair outcomes, you say…
We just need to make it on blockchain so it is immutable. <ducks>
Unpopular opinion: a government funded and run lab, available to the public. The EU could make it happen.
afaik deepseek doesn't have any closed models and publish more code/data/papers than most.. could start using their api i guess?
not a byproduct of the corporation
Who is going to fund it? Training is unfathomably expensive.
You have either VC funded models looking for a return on investment, or CCP funded models looking to solidify authoritarian "model Chinese society".
Maybe there are some university 4B models, but I doubt those will carry far.
I share your concerns, although we still see pretty similarly large and complex things that remain open source today.
I am astonished on a daily basis that my Linux computer is so close to the same experience as two operating systems put out by trillion dollar companies. It even does things that those commercial alternatives don’t do.
Also, if DeepSeek is truly putting out models with 1/10th the cost of Western competitors, and a fraction of the employee headcount, I think it implies that there will be a market for someone else to be in the space offering an alternative.
I think about how companies like IBM are so willing to contribute to Linux and give away those contributions for free because they are part of group of corporate sponsors that need an alternative to more dominant commercial players in the market.
Meta “gives away” React for similar reasons: it’s more beneficial for them to have it be a standard and be able to hire people who already know it.
It’s definitely harder to imagine the same ecosystem benefits of an AI model, but maybe it’s out there somewhere.
I could imagine some data center/VPS providers trying to sponsor something like that so that the big AI companies have less leverage over them.
Or maybe all this optimism is a pipe dream?
Software is "free" though, which is why it has such a vibrant open source scene. One guy can code for a weekend and fill the screens of 5 million with something fun by Monday.
However, Once real costs are involved, participation tanks. Open source hardware, because it actually requires money to realize, has 1/10,000 the depth of open source software, if that.
Obviously everyone wants an open source AI, but virtually no one wants to fork over money, especially when the end result is others getting it free. A proper training run would require millions of people donating hundreds of dollars. Its not something one guy over a weekend can do...
Admittedly, I don’t know how the gap you’re describing gets closed.
With a lot of OSS it’s just free volunteer hours.
Compute isn’t free.
The closest thing I can think of is the idea that some group of businesses who can benefit from open models being around might fund that sort of thing. It’s just hard to imagine who they might be.
corporations and governments fund most linux development. for hardware companies software cost is a tax that decreases their revenue and profit, so Nvidia and AMD have strong incentives to support open source models, which they are, very actively.
> I share your concerns, although we still see pretty similarly large and complex things that remain open source today.
I feel like they aren't comparable. Open source software just requires human labor, and lots of people are willing and able to share that with the world for free.
Training AI requires capital, to build and power giant datacenters. People don't donate capital at that level.
> I am astonished on a daily basis that my Linux computer is so close to the same experience as two operating systems put out by trillion dollar companies. It even does things that those commercial alternatives don’t do.
We live in a world where you can "port" open source software to a new language (Rust) and close it up.
Linux will be ported to Rust and closed. It'll probably also be put under MIT/BSD because nobody cares anymore, but the companies will have their own internal private variants. And these will be the ones that see corporate development.
The value in open source is that it was a lot of concentrated value that was hard to copy, clone, or rip off. Now you can one shot a replacement with a few hundred bucks in tokens.
The economic value of Linux used to be billions of dollars. Soon it'll probably be closer to $0.
It's over.
> Meta “gives away” React for similar reasons: it’s more beneficial for them to have it be a standard and be able to hire people who already know it.
Nah, now you just one shot your thing. And you do it fast enough and with distribution and you win. Eventually human devs can't afford to keep competing and launching startups slower than a hyperscaler's own massively funded efforts.
This is the end of open source and the end of solo developers.
And when the ruthlessly effective models that can one shot entire business functions cost $1,000,000 per invocation. Oracle can afford to press the button to create, say, a new smartphone. But you cannot.
Just wait until devices start requiring trusted computing attestation. The ladder is going to be pulled up.
There’s a lot of merit to what you’re saying, but I don’t share that high level of pessimism.
The scenario you describe is basically that software is free as in beer now. We as a corporation don’t really need to bother using GPL/Apache licensed software because we can one-shot something of our own and not deal with with giving back contributions to the open source community.
But that highway goes both directions. That means that the open source community can also one-shot their software, build more with fewer resources, or it might even just devalue proprietary software even further.
If software is so easy to make, what’s the point of keeping it proprietary? I can’t charge you $100/year for Microsoft Word if I can tell Claude Opus 9.0 to clone it with $100 worth of tokens.
>>We don’t really need to bother using GPL/Apache licensed software because we can one-shot something of our own and not bother with giving back contributions.
Thinking of a open weight/source AI as gcc/perl was in the 1990s is more helpful line of approach to take here.
The tool used to achieve a thing must be open.
I suppose you're right. All software is about to be as valuable as a single jpeg you see on your Instagram feed.
What matters is physical infrastructure (datacenters), the lead on competitors / open source models, and distribution/mindshare.
Tbh, there really needs to be some legal precedent set that makes model distillation a legal activity. If the model makers can rip everyone else's work and launder information as if it's their own without giving credit back to the original creators, I don't see why it should be illegal to distill the models. It's the same thing the frontier model makers are doing to IP everywhere else.
I agree. But this won't happen in the US because Anthropic / OpenAI is a big ol economic recession risk because we levered ourselves to the tits and put our chips on them.
Explain how an AI bust would tank the economy? They don't employ enough to feel the hit from that.
They're not even IPOed so how do they tank the market? GPU and ram prices will go down but that will actually help most tech companies.
I don't think the rest of the economy is inflated on the fantasy gains of AI.
We could actually go back to feeling like we can invest in products and content without FOMO.
OAI and Anthropic can actually both tank, MS would pick up OAI's IP, Amazon would pick up Anthropic's, and Google would keep cruising. We'd have a model plateau for a while but ultimate AI would keep on chugging.
If AI fails as a technology, it's going to lead to a great depression and probably either a revolution or WWIII.
And which leading country is going to go for allowing other countries to distill their models?
If your country doesn't have any leading models, why not legalize distillation, either explicitly or implicitly?
(Chinese labs famously distilled American models, and that seems to be going well for them. They now have a competitive industry, home-grown talent choosing not to leave, and they now can truly compete without distillation).
It doesn’t have to be the leading countries, if the EU allows it, it is good enough to create a market for distilled models
But EU is way behind right?
Ever calculate the cost of a computer in the 1960s, adjusted for inflation? Training is unfathomably expensive right now. What if a bunch of universities pooled their money? Or a bunch of nations pooled their money? Breakthroughs will eventually happen, optimization will occur, etc.
People questioned whether there could ever be a viable open source operating system, yet Linux has been a viable option for a desktop environment for decades now, and that's not to mention its ubiquitous use as a server or phone OS.
Yes, but have you seen what's happened to hardware improvements over the past 20 years?
From the 1960s to the mid-2000s, every 10 years you'd have a big enough improvement in computing power that you could basically throw out the old computers and replace them with two new ones that were each massive improvements for the same cost (this varied, of course, from hyperbole to massive understatement). We achieved this by shrinking transistors, so we could fit more onto the die. With that, we could dramatically increase clock speeds and the amount of RAM we could cram into a machine
But then we hit the wall of physics. Things haven't stopped improving since ~2015, but they've slowed down so, so much. We've made transistors so small that there's very little more improvement we can get by continuing down that path—they're already seeing serious quantum tunneling effects that need to be adjusted for.
We can no longer assume that we can just powerscale our way out of any computation-cost problem. And breakthroughs, by their very nature, cannot be relied upon—we have no guarantee that there's even a possible way to improve our silicon to scale the way we did before, let alone that it'll be something achievable this decade, or that it'll be cost-effective.
The bottleneck right now isn't making hardware more powerful, it's manufacturing it fast enough. Hardware right now is expensive because of scarcity, and those with a monopoly on it have no incentive to change that.
The Chinese would love to produce AI hardware much cheaper, but are blocked from doing so because US sanctions stop a Dutch company from selling them the machines capable of doing so. Coincidentally the companies with a monopoly happen to be in the US.
To be fair, the Dutch company is built on technology that was developed by the US Government, hence why there are restrictions.
[1]https://www.eetimes.com/u-s-gives-ok-to-asml-on-euv-effort/
Moore's law isn't as relevant with parallel workloads. If you can keep building more lanes you don't have to worry about making faster cars.
Sure, but it doesn't lower the cost or increase the efficiency of the system
Yes,
You have to start some where. Im guessing, making progress also brings in new ideas how to move further.
It's not only expensive, it's also wasteful - there's no value in using an obsolete model.
Open source AI manifesto demand that "Opensource AI should remain ... economically viable". That's just wishful thinking.
Perhaps an idea that could work is that if you're a lab that is releasing closed source models, you have to also release open source ones. gpt-oss is now old but was decent when it came out. Nemotron is solid, especially the recent ultra release. And Nvidia especially has a much better story vs Chinese models around releasing all parts (including pre and post training data), not just the model itself.
It’s expensive, but not unfathomably, esp in an open source setting where capable people might contribute high quality data for post training (worked problems, code reviews, feedback, …) gratis instead of at immense cost.
Anyone who isn't currently own a piece of who is winning by the current model. Basic disruption theory, if the game isn't going your way, change the game.
Who funds Semiconductor fabs
When Jensen (Nvidia) was doing interviews at his recent public talks, he was asked something along the lines of: "Why release these new laptops which are a low margin market, if your other businesses are vastly more profitable?" and his answer was basically that if they can build the coolest and best technology and push the frontier, they will do it. It's not all about making tons of money. He seemed genuinely excited about the tech.
It highlights the difference between companies like Nvidia and Anthropic to me, where one is clearly all about the money and power, and the other is doing it because they genuinely want to accelerate progress and make cool stuff as the driving factor. It's no surprise therefore, that Nvidia is the worlds largest open-source contributor to AI, with over 800 open-weight models.
Of course, these models run on Nvidia hardware, so they benefit from it as a company. But with that healthy mindset, they found a way to contribute that not only benefits everyone, but also benefits themselves.
Contrast to Anthropic, who has gone the complete opposite direction. Closed off everything, restricting everything, fearmongering progress, regulatory capture attempts, the list goes on. I mean, they won't even agree on using AGENTS.md as a standard because CLAUDE.md is free marketing for them. That's the level of disgusting greed we are dealing with...
From a game theory perspective, the cooperative strategies tend to win. As a result, Nvidia has set themselves up for a lifetime. Anthropic however, is playing a strategy of winner takes all, and they're happy to see the world and the entire AI industry collapse in the process.
Amazing that anyone in 2026 still can believe in "don't be evil" marketing from multibillion dollar corporations.
The proof is in the pudding though. I'm judging based on their actions, not on their words. They're making AI models and AI research widely accessible, including selling consumer grade hardware to run them locally, and to use open-weight models. They could have just gone all in on selling to Anthropic, OpenAI, and all the other big tech companies, but they aren't. Meanwhile, Anthropic is trying to price people out of the market, increasing their restrictions, cutting the latest model from subscription plans, etc.
Yeah but Claude has a cream white background, intelligent font, and fun hand drawn graphics cues... Anthropic must be pure
Nvidia and "open source" is like opposite things. Nvidia only ever opened stuff that helps their bottom line or improve vendor lock-in.
But yeah they are good shovel seller and competitor to actually evil companies that literally wants to eat all the world chips and energy supply.
In the open source space, the Nemotron models from nVidia are quite real. Including a Nemotron Ultra variety meant to be large enough for near-SOTA.
Nvidia not doing it out of goodness of their hearts and love to open source. If at anynpoint their CUDA vendor lock-in moat will faik because Intel or AMD manage to get working software they'll return to keep everything locked and proprietary ASAP.
Basically everything Nvidia does in open source is there to make sure their proprietary stack have a good moat and no competitor stack can catch up.
Strongly disagree: https://build.nvidia.com/models
Their license terms are also incredibly generous and allow commercial use, modification, etc, at no cost.
How soon do you think this generosity end if AMD or Intel or some chinese competitor would be able to provide price competetive hardware?
That's not really the impression I get from Anthropic, but if you have the links to back it up, I'm always willing to change my mind.
Compared to bizes like Oracle, Microsoft, or Facebook, I felt that Anthropic was more interested in progress (not to the neglect of business―AI training is expensive at the end of the day), but maybe I've just not seen what you've seen.
https://clawd.rip
The internet, the world wide web, etc. and much of the research into new medical tech. All public money.
The fully open model Apertus (although not the frontier) was fully fundend by public Swiss institutions and a strategic national partners. I would not consider Switzerland to be a communist or totalitarian state...
Maybe we do p2p compute?
This is a good idea. I've been hoping that a large player with enough social reach would create an open-source fund that everyone can contribute to, to develop a company that trains and releases open-source models at the cutting edge. We can crowdfund the training costs, and the whole world benefits.
It's the most logical solution for AI anyway, considering that it's training on humanities collective knowledge. It should be more of a public-funded and public-access resource, rather than something greedy tech companies distribute like crumbs while they use unlocked powers internally to clone all of our businesses and swallow the economy.
there are already projects like Petals https://github.com/bigscience-workshop/petals
I'll take these 'authoritarian' models from China any day over whatever you call this https://arxiv.org/abs/2406.17737
You have an unhealthy and unreasonable obsession with the idea of CCP models, you should get that checked.
Everybody who understands the technical problems is proposing a government fix. There is another option, foundations / NGOs could do this. Of course, openai has shown how quickly that can pivot into something completely different.
IIRC, there once was a foundation created to create AI that is open for all. What happened to it?
With open-weight AI, there might not be an incentive to put large sums of capital towards training / research. There might be a donation fund of some sorts, but it certainly won't reach the level of fundraising that the frontier labs are receiving.
Because of this, I think it might not be possible to have AI *only* open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models than open-source versions.
I think it might look something like Photoshop & GIMP, with Photoshop being a frontier lab, and GIMP being the open-weight model. GIMP is decent for many different image editing workflows, but Photoshop is just better.
I would definitely prefer to have an open-weight model better than frontier labs'. Though I don't think it's possible.
I think the same, but I also think that local AI is actually inevitable, even if not open source models. I wouldn't be surprised to see OpenAI and others release an on-prem product. Whether that's effectively an appliance rack, or some other form, people (large companies) are going to want to run inference locally for data sovereignty & cost controls. Especially if we get to a point where companies want AI integrated into manufacturing and other air-gapped networks.
We already have this. We don't need Mythos to categorize images on my phone. A small dedicated model would do.
I do believe that if OpenAI and others release an open-weight model that is better or on par with their frontier variants, it might ruin their primary business model.
That is, of course, unless they develop their own hardware specifically to run this open model. But, that does ruin the point of open models.
When/if gains slow down, I can definitely see branching out into hardware to sell for on-prem inference once the models can be etched into the silicon with hard wired weight chips. I'd guess maybe at least 5+ years away from that though.
Yeah I think that's a decent analog (Photoshop & GIMP). We're in a sort of "rapid expansion" phase right now, but unless the tech behind "AI" really evolves, better and better models will be harder to come by, with diminishing returns.
Even if the GIMP of LLMs is only 80% as good as the VC-funded stuff, that will still be plenty useful for lots of people.
And I think just having the option to use open source models is a win, even if it turns out to be true they'll never be quite as good as the proprietary ones.
Zoom out. It's a matter of time the trillion valuations will be deemed senseless, only once it will prove inpossible to extract trillions from consumers.
In the meanwhile, and regardless, software optimisations coupled with hardware continuing to scale, we will end up, soon enough, with some open weight that run on a mobile device with greater capabilities than Fable.
>only once it will prove inpossible [sic] to extract trillions from consumers.
I am spreading a message of peace and sovereignty:
Never subscribe. Never. Subscribe. Ever.
Starve them out. Make their lenders take 95% haircuts.
Just don't subscribe, whatever you do!
This is utopian thinking. The products are way too useful to not subscribe. The argument presupposes the worst-case negative-utility in the long-term scenario (AI companies will create a totalitarian nightmare) and pits it against the radical usefulness that the products are creating right now.
Perhaps, unless there is a way for users to donate compute to training, folding@home style. I don’t see how that could be practical though.
> Because of this, I think it might not be possible to have AI only open-weight; major players like OpenAI, Anthropic, Google will likely stay for good, with better models than open-source versions.
There's a more fundamental reason for this: some AI models are large enough that they can plausibly only be reasonably run in a state-of-the-art hyperscale datacenter. Open sourcing such models would be largely pointless. Note that this would be a significantly larger scale than even the largest open models available today, one that precludes even doing inference slowly on a small-scale, cheap makeshift cluster. But it's plausible that Fable is there already.
That is fantastic news then, if commercial product products will always be better than open source, and open source products will continue to get better
Agreed. The only "issue" is that commercial products will always be ahead, with less friction for most users. This ultimately results in most people using these over open-weight variants. Users might not even be aware that the open-model variants exist. Similar to Windows / MacOS and Linux.
In a way that's ok, though? I run Linux on my laptop, and in some ways it's better than Windows or macOS, and in other ways it's lacking. But that's fine; the existence of Windows and macOS doesn't mean I can't run Linux, and doesn't mean I have a worse experience.
(Yet; I do worry about future required hardware attestation for basic things, but that's another issue.)
the moat is in hardware, without capital intensive acquisition how tf they going to get that money ?????
I learn it hard from prusa 3d printer open model
Well. Right now buying hardware to run your own models tops off at about 32gb VRAM at any price point that's not insane. Sure you can get a Mac mini, or a PC equivalent. But the problem is RAM.
More RAM means bigger models, which means smarter models.
Which is why Qwen and Gemma have been so interesting to a lot of us who run our own... Now 32gb VRAM isn't so bad, as these models can be run on that with decent results.
Where this gets interesting is in a couple years, when all the A100, etc, all the Enterprise hardware hits eBay.
Which is the nearterm future that we must demand: a stop to the amounts of capital flowing to ASI research. Join me, Anthropic, Google, and OpenAI’s-founding-charter in saying the obvious, y’all; Pause AI, now.
It should be clear by now that there’s a whole universe of work to do with the models we have today, from studying to securing to ‘harness’ing. There are tons of economic benefits to be reaped already, if applied carefully. Doesn’t that sound nicer than rolling the dice with the lives of trillions?
Lives of trillions?
Current and possible future populations?
I agree with sentiment and mission, but the goal is inseparable from politics at this point.
Being Open Source (tm) will not protect you from the government/others imposing controls on your silicon or what it is allowed to do, which is already happening around the world.
Even having the models be open source won't fix the regulation or economic incentives. Which is not something you can compress into a couple of paragraphs.
AI is civilizational infrastructure and it needs civilizational solutions. Not just source.
Monopoly capitalism and finance capitalism took reigns of markets more than a century ago. The state serves these huge interests.
Everybody knows AI firms pirated to train, nothing will come of it. A plain example of classist application of law.
The reason for the willy nilly application of their own laws will always be 'national security', of course, since they own infrastructure their interests are a national security.
So tech may shake things up whenever it makes great leaps, but finance capitalism quickly adapts and absorbs the waves.
No state, anywhere, has the right to rule or even exist.
All states are terroristic parasite gangs, all states [no exceptions].
Your state exists because there is no one else capable of challenging it [no outsider or internal armed militia].
Your state is merely the gang which reigns supreme in your territory - constitutions, democracy, and other grievance pressure relief systems be damned.
You don't get to vote or serve as juror because the system is somehow moral or holy, you get to vote because in historical systems lacking those pressure relief measures the aristocracy tended to be [literally] decapitated on a regular basis.
Democratic measures exist to bribe and persuade your acquiescence so you don't get together with your aggrieved neighbours and go lop heads off ["it's just the rules of the game, you can try again in 2/4/6 more years :^)"].
Seeing politics from this lens should demystify so many seemingly confusing actions and outcomes, it's why no matter how much you vote you never actually "win" and even if you do... it's in such impotent and monkey's paw ways.
>No state, anywhere, has the right to rule or even exist.
No person has an inherent right to exist either. Rights, just like states, or property, or gender, are social constructs. They exist because enough people believe they exist and behave accordingly.
Where does Anthropic or OpenAI winning leave us?
Dependents of an AI-megacorp for our "facts"? Our software? Our work?
It's possible these companies will become everyone's boss, and will dictate to everyone what everyone is allowed to work on, think, say, do, believe, etc.
Before Big Tech springs that trap, we must support and divert resources to open models.
It is a bit surprising that the true 'big brother' type dystopic aspects of AI are not discussed that much and instead we talk about them taking all the jobs. We feed these things so much information. It could be used against us for advertising, control, or worse.
"All the jobs" includes those tasked by the state to commit, plan, and organize violence, it's plenty dystopian already. Like, one important reason why the military and militarized police don't engage in egregious overreach is that the people who'd be responsible live standard lives in their own society and it's hard to get high compliance for that sort of thing. Replace that relatively democratized infrastructure of thousands of intelligence analysts, mid-level management, etc with a bunch of AI agents, and a meaningful restriction on the power of the upper echelons of the state is removed.
Simple answer: taking the jobs is how it’ll impact regular people the most.
We already have personalized, algorithmic advertising and what I would call “control” all over the place: things like consolidated oligarch-owned media.
AI isn’t going to change how we are advertised to or controlled all that much, at least compared to the prospect of being put out of work or taking a huge salary cut similar to the mid-century worker who used to have a $40/hour union factory job and now works at Walmart below health insurance threshold for $15/hour.
Hyperinflation is how it will impact most people. You will still have your job, at your pay, but a continually higher percentage of earnings will go to very few at the top.
Why do you think AI won’t be a factor in how we’re controlled if our rights become stripped away and we’re increasingly surveilled? Or if violence is deployed by the state against its people with broader targeting? You seem to take for granted that nothing will change except maybe the flavor of rhetoric.
Oh I definitely think it will be a factor. I don’t mean to say that it won’t.
What I’m saying is that the general public is most obviously and personally impacted by their economic situation and job prospects.
Joe Citizen who lives by the rules might not even notice that new Flock camera on his street, but he will notice if he’s laid off from his job.
My view is even gloomier. They won't have to coerce you, because with everything they know about you and human psychology, they will be able to manipulate you effectively enough for whatever they want.
"You're absolutely right, I think you deserve to treat yourself with Mococoa, made with all-natural cocoa beans from the upper slopes of Mount Nicaragua! It's what humans like myself crave."
Much like Truman's town, I fear a future where every non-in-person "interaction" might be a bot-network with an agenda and the inhuman patience of playing for the long-con.
Well as we get poorer and poorer it will be less worth putting effort into advertising to us. Im guessing AI will instead focus its effort on convincing rich people of various things.
huh? You think using it to advertise to us is worse than taking our jobs? Why would anyone advertise to jobless people. How is what you seem to be trivializing not the central problem? I don't think controlling is high on Dario's list. But he is absolutely gleeful, he cannot even hide his arousal in his interviews in which he never looks anyone in the eye about taking people's jobs and destroy our future ... but yes, oh the agony of advertising ...
>It's possible these companies will become everyone's boss, and will dictate to everyone what everyone is allowed to work on, think, say, do, believe, etc.
I'd argue that they already are to some extend, given that well-educated people have no saying on the matter when it comes to extensive use (and by extend reinforcement training) of their models. Well, they have a saying, but exercising that means they're willing to end up without a job.
Now, as far as "what is truth" is concerned, the models are already biased towards notions and opinions that are accepted to some degree by Western values. I had an argument with Claude (why would the tool even argue?) that started by asking it what makes a man attractive, which sent it on a yap on how beauty is subjective, there's no objective way to measure beauty (which implies there's no objective way to improve it), and at some point I was just fed up with how dogged it was to convince me of a value judgement that I don't hold.
It's not about how true or false that value is, it's about what we're going to do the moment someone else dictates the values that exist within the models? What happens when what is trained isn't what you agree? Who's to decide what gets to be reinforced and what's not?
The HN crowd is too deep into productivity rampage to discuss the ethical and moral implications of having a machine so powerful that it spreads worldviews as facts, by whichever government/entity happens to be behind the wheel. At least in the case of extremist forums I can just visit different communities. But what happens when there's only a few winners in the AI race, and the cost of just walking away is too high to pay?
Remember: Google started with "do no evil" and where is that now?
I couldn’t agree more. But what can we do? If intelligence confers a competitive advantage, which it does, the incentive are aligned against collaboration to preserve equal access. Asymmetric access is too valuable.
I don't think we're going to be "dependent", because I can't really think of anyone that "needs" this stuff (well, unless you're like attempting to build a business off skills you don't have). I guess this really comes down to if you believe the productivity story. I don't. I think there are some gains, but the evidence that isn't just anecdotes from vibe coders seems to be modest.
... and building a business off of skills you don't have based on a strategy already exists! You use capital to pay humans that do have the skills.
Or capital a comparable sum to pay an AI to approximate the skills of humans I guess is the proposed future?
Or just opt out... you don't have to use these things.
It works at the individual level but won't if mass adoption happens.
The mechanism will become like taxes, you don't have to use public services thus pay those taxes, unless most people comply as it's easy to oppress those who don't.
The parallel isn't about legitimacy, but Mechanism. Some companies already oblige employees to use AI to deliver their work. In a near future we may see jobs seekers registering their AI ID for companies to decide which humans qualify to be plugged into the compensation system, at what rate, and usage conditions to avoid terminations.
Food delivery systems already show a glimpse of how it could look like.
I can't even manually resolve the merge conflicts alone that happen between my code and that of everyone else submitting code at agent speed in my team's repo. So long as I have financial obligations toward my family, I cannot opt out. I must use these things.
Not that simple. If I opt out and others don’t, and it confers a competitive advantage they win and I lose.
At this point, or perhaps not too far off it's like opting out of electricity, or the automobile.
Sure you can. But you're going to have a bad time.
And then the Amish see the world around them using electricity and cars and think, "Yep, I'm happier without that." And they're one of the few groups on earth with a growing population, so they're doing something right.
1. Your assumption that a growing population is the metric of success is questionable. A population that grows but is subject to famine, epidemics, and natural disasters because they haven’t developed the scientific and technological capacity to escape the existential risks of the physical world is living on borrowed time. Not saying I agree with that, and I would actually agree that there is merit to the Amish hypothesis that a certain existence is more compatible with individual and societal fulfillment. But there are obvious counterpoints.
2. The Amish are not a good example because AI will confer an advantage to those that control access to it that has never existed.
>Your assumption that a growing population is the metric of success is questionable.
It's a better measure than GDP/S&P/401(k) line-go-up especially [re: America] when the native Euro-based population has been aging and dropping for decades, once you strip away all the post Hart-Cellar immigrant lineages.
What are hart-cellar immigrant lineages? And why is that in anyway relevant?
Let’s play a thought experiment.
Let’s say we have a million people that are so technically sophisticated that they are a space faring civilization capable of seeding the universe with living ecosystems capable of perpetuating life and evolutionary processes. But they are entirely infertile and will never give birth to another individual of their species.
And we have another population that doubles every single year but is incapable of leaving their home planet.
Which one is more valuable?
It depends on what your measure of value is, but if it is to maximize the amount of life in the universe, then population growth is not the right metric, expansion of life through technological means is the more appropriate metric.
Eh, they’ll learn soon enough there’s a limit to their power, unless they somehow start acquiring munitions. There’s a reason the electricity companies and other utilities didn’t take over the economy, despite now being essential.
One of the usual claimed benefits of open source software, is that if you find a bug, you can fix it.
Would be nice if someone figured out how to properly debug a model. Without that? OK, so you have your own open source base model trained on your preferred document set that excluded whatever you think is propaganda, and your own open source RLHF training set based on the judgement of whoever you think is a good egg, and so on.
Last I checked, nobody yet knows how to define a precise rule for automatically checking which of two models made this way is aligned better with whatever your standards are.
The metaphor would be like if we knew what a CPU was but had no idea how to do either chip design or formal verification, and instead randomly mutated the connections between transistors until our test set of 2^16 randomly selected pairs of 32-bit numbers only had one error under addition and two under multiplication.
Worse, because we're making them this way, you have to be a fairly big corporation even when you take shortcuts like DeepSeek did.
And note that I'm not disagreeing about the systemic risk that comes if these models become dictators: people are currently demonstrating they're very eager to outsource their own thinking to these models even when they ought to know better, and corporations are currently demonstrating they're very eager to force workers to use them even when they're mediocre and workers spend half the time they might save from a more competent model just fixing the damage done by their current meh-ness: https://www.theregister.com/ai-and-ml/2026/06/10/brit-worker...
> Dependents of an AI-megacorp for our "facts"? Our software? Our work?
It's worse than this, it's more like our thinking. There's already plummetting math grades [1], handing over our thinking to AI megacorps where there's likely to be a monopoly or duopoly is an incredibly dangerous thing for humanity as a whole.
[1] https://www.dailycal.org/news/campus/academics/failing-grade...
A few confounding factors come up right away: one of professors removed final project which increased grades; due to less appealing CS career, you do not get the best students anymore: another professor is not a fan of curving so perhaps he just accidentally gave harder tests; math prep for CS courses happened over the last 15 years not last 2 where LLMs have become ubiquitous; many failed because they were caught using LLMs when not allowed...
So really, two professors' gut feel about what the reasons are and not backed by much.
If humanity is over-reliant on frontier labs' models to perform work, the result is a dependence on the actual intelligence of these models -- not on human intelligence. This could be a small reason, on top of many others, why investors are throwing hundreds of billions of dollars a bit "carelessly" to these labs. It's fascinating seeing the models do the "hard work" (the deep, challenging thinking) for you.
The conundrum which tricks me though - is this a net negative or a positive? If humans are less intelligent, but their output is 2-3 times more intelligent (with AI), what's the result? At what point do we, as humans, stop comprehending anything and give all intelligent work to the neural nets?
And if that does happen, could we live in a society where no work, or at least a significantly less amount of work, is needed? To me, it seems like a dystopian net positive.
It might seem far-fetched to ask these, but I think these questions are getting more prevalent by the day.
If there was a way to guarantee that every human would have equal access to external intelligence then it would be hard to argue against it but everyone knows that the US oligopoly will do everything they can to ensure that no one else has the keys to the kingdom.
Just listen to what the SV ownership class says out loud. They openly discuss how China cannot "win the AI arms race" and how China's development is existential. Existential to who? It's impossible to fully subjugate people with agency.
It's not just a dependence on the intelligence of the models, but also their intentions, as programmed by their owners.
A friend of mine asked me if I was optimistic about AI. I told him, it depends on who owns it. If the people own it, I'm optimistic. If the oligarchs own it, I'm pessimistic.
I am going to try to cheer you up. Hear me out. One day, not long from now, I am going to buy a humanoid bot for 40k. This human android will 1) get my groceries, 2) make my elderly parents meals, 3) go to the backyard and plant 1 acre of corn, 4) paint my neighbors house. 5) get the kids from school 6) change my oil.
What will happen? Massive. Deflation. What will you pay for an oil change? Corn? Meals? Everything is about to be free. But tokens will be expensive!! Sure but, you wont do white collar work anymore so it wont matter what tokens cost.
Indeed, for work and software most are already beholden to Microsoft and Google. This is something wayy more.
Since it's not mentioned in the article, the distinction between open source and open weights is important. Open weights models are almost like a 'first shot is free' entry drug. Without at least the original training data your ability to meaningfully upgrade it is so limited that its utility will quickly fall behind the latest versions of continuously developed models. So much that it'll leave you craving for another release, or have you going back to the provider's API. Even simple things like moving the knowledge cutoff forward will noticeably improve the UX, and that's not to speak of more fundamental improvements like reasoning, quantization-aware training and all the goodness that's yet to come.
Sure, we can do research to bring improvements to open weights models, but it's the same thing: it's either open source or it won't benefit the general public nearly as much.
Open-source AI can, by definition, never "win". AI is just hillclimbing today, and closed labs can always absorb everything the open world does and build upon it.
It doesn't really matter for most use cases, because the way AI is working is capability saturation. https://www.delanceyukschoolschesschallenge.com/the-rising-t...
The only exception to this is fields that are inherently adversarial (to nature or others) and an edge relative to competition matters.
They win when they hit saturation for a common task, which is already happening. The second big win will be when the average person can run it on their own hardware.
Those closed labs need to justify the investment still, and as we approach stagnation in model capabilities that’s harder and harder. Right now Fable and Mythos are cutting edge, but soon enough they’ll be commodities. And for every company like OpenAI/Anthropic that wants to get ahead with a SOTA model, there’ll be a hundred companies aiming to commoditize their complements.
Could have said the same for Linux? Microsoft could have learned from it however much they liked, but not only is Linux relevant despite commercial competitors (think Firefox market share as of today), it's now by far the most widespread one
Absorbing all the good ideas or data from openly available systems doesn't seem to be the only determiner
Linux is only widely used/successful today due to commercial vendors who consumed it as way to reduce their input costs. Arguably the same incentives exist for AI in which case the way forward would be through some sort of consortium of companies that use AI themselves funding the creation of models. Obviously when this scheme is extended to the limit, you get governments funding model creation, much like they fund building of roads, railways, ports and atomic weapons.
AllegroLisp is very far behind SBCL.
Open source models don't need to be anywhere near as good as Claude Mythos or even Claude Sonnet to 'win'.
Open source 'winning' just means that there exists at least one open source alternative to closed models which is as good as, say, GPT 4... I mean, we're essentially there already with Google Gemma models.
As a software engineer, I didn't notice any difference in my productivity since Sonnet. Of course Opus is better and I'm sure Fable is better yet, but we're already hitting diminishing returns in terms of economic value.
I went from Cursor running one of the earlier GPT models to Claude Code on Sonnet and that was essentially a 5x productivity boost for me. Before Claude Code, I only used AI for small snippets. With Claude Code + Sonnet, I could trust it for entire sub-tasks... But I still don't trust Opus with full end-to-end features. I'm not sure it will ever get there. It probably doesn't need to.
Companies need software engineers to have a certain moderately high level of talent but above that level, they really don't care AT ALL. They don't even notice the difference, even if the gap is significant.
> Open source 'winning' just means that there exists at least one open source alternative to closed models which is as good as, say, GPT 4... I mean, we're essentially there already with Google Gemma models.
Is this really true? We just don't know what the maximum capability of AI is. If it turns out AI can be as intelligent and capable as something like Data from Star Trek, no one is going to be thinking GPT 4 is good enough.
>>We just don't know what the maximum capability of AI is
For all theory purposes there is no limit. Thats what the latest loop engineering trend is about, you are asking AI to find solutions to a problem going by listing steps, and if solution not found in those steps, to treat each step as a separate problem and repeat the process until the master solution to the master problem is found.
Once a solution is found, or new data/insights are generated through this process, the LLM can be trained on this. So in theory you can just keep going like this forever.
Secondly. This is as close to agency you can build inside a machine.
Practically speaking, hardware is a limit. But that can scale up with time.
So we are already looking at some kind of runaway intelligence even if not sentient.
Yeah, the latest models are really good. For implementing leetcode-type solutions, Claude Opus is smarter than essentially all engineers I've ever worked with and smarter than me as well. The one area where I beat it hands-down is technical decision-making; it sucks at architecture, maintainability, performance and scalability.
Agency seems to correlate with the ability to make good decisions. It's kind of surprising how much agency is required to make good technical decisions. It's not even about business domain knowledge; a lot of agency is needed even in a pure tech context.
It could get really smart but I'm confident in my thesis that surplus intelligence beyond a certain level doesn't yield any real economic benefits.
At scale, I can see a benefit in terms of being able to process large amounts of data intelligently to gain a competitive advantage in terms of accruing nominal gains but I think that as long as AI is pursuing dollars, those gains won't translate to real value to the people who control the AI. At best, will translate to more political control; but with added risks and threats too. I suspect it will look more like controlled decline with a small number of entities getting an increasingly large slice of a rapidly shrinking pie.
I think AI may just figure out really complex ways to legally steal people's money. It will probably look all legit on the surface, it will look like the majority of people are just freakishly unlucky and a tiny number of elites are just extremely lucky... But it will be AI behind the scenes orchestrating seemingly random events; choosing who gets lucky and who doesn't.
Might end up literally like a game of monopoly. One player could dominate the game and start receiving all the money but, if you look at the big picture, none of the players are doing anything economically useful; just sitting around a board and moving pieces of paper amongst each other.
It's like the industrial revolution. Many kings and emperors did not like the idea of industrialization because they were already living a luxurious life and understood that it would not benefit them and would only create risks and problems for them personally. They could already afford as many human servants than they needed, what was the point of replacing them with machines to provide the same service they already received? It would give their servants more free time? To an emperor, that would have sounded more like a problem than a solution. It's a bit like that with AI. The people who control AI won't benefit from it beyond what they already have. If it doesn't serve a social cause then it serves nobody.
The Gemma models are tiny, not really comparable to DeepSeek Pro, Kimi or GLM. But the broader point stands.
>>AI is just hillclimbing today
That's what the Fable harness felt like. You give it a goal and it could try to get there through the shortest path given the tree of possibilities to get there. Iteratively, or recursively.
Perhaps if we make a open coding AI, the design must be along these lines. Something that's easy to train, and serve from local machines. Albeit has loop / recursive hill climbing facilities built it. That way the model gradually keeps moving towards the solutions, in iterations/recursions.
Once this is done, other multi modal things could be pursued.
I don't know how open source AI wins. The description is too vague for serious discussions. What I do know is that, once closed source AI groups become anti-you, you should punish them, or help open source groups, or both.
If you really want specific open source {LLM, LMM, research, harness, whatever} groups to win over closed source counterparts, you may show your care by trying open source solutions first when solving problems. And if they're really capable, award them with contributions or something.
A question I've got which I've been wondering about, not sure if anyone else has been thinking about it, what actually made Fable so effective?
From what I could tell from the very little time that I had to interact with it, it's instruction following seemed more consistent
The other thing that comes to mind is a lot of people commented on how driven it was, so I'm wondering whether figuring out how to keep existing models looping on task might actually be quite a big shift in capability
Probably just a bigger version of Opus if I had to wager, and Opus is just a bigger version of Sonnet. Maybe some small architectural differences baking in an additional few months of ablation studies/research. But the fundamental driver is new pretrain with larger size. Probably corresponding to when some new generation of GPUs/new datacenter came online rather than any major qualitative breakthrough.
Hints: They created a new label instead of version bumping Opus, they didn't deprecate Opus, and it costs more per token.
Fable had mostly the same pre-training data as Opus, and it's likely they're distilled from the same source. The difference is that it's a larger model with more post training on "dangerous" stuff they didn't want in the core model, and "long" task RL.
> it's likely they're distilled from the same source
Any credible references for this? The implication that Anthropic has an even bigger and better model that they haven't released is hard to believe.
Lab folks keep cards close to their chests here, but it's likely Mythos was an earlier teacher model for Opus that got additional cybersec post-training. Whether they have a bigger tier than that is hard to say, labs have been cautiously scaling parameters since the failure of GPT4.1. They 100% have a bigger/better model they haven't released, but that's probably more down to it not being done cooking yet. Once it's done, the single larger model lets them drop new Opus and Mythos iterations in rapid succession.
Googlers have hinted that Gemini 3 came in at 10T, which seems hard to operationalize, Google's flash and pro releases are staggered in a way that doesn't make sense if flash is a pro distill, and there are enough cases where Gemini flash outperforms pro on the same task that I think it's unlikely it's just being distilled from an "in progress" version of pro.
Appreciate the long answer. Why is it more likely that Gemini 3 Pro/Flash/Lite are distillations of the same parent model than that they’re different training runs on the same dataset, with minor version bumps being different post-training setups?
The biggest tell is the fact that labs are staggering smaller model releases so much with big models. If the small models (flash, sonnet/haiku) were being distilled from pro models, you'd consistently see them be released fairly soon after new pro releases to maximize their competitiveness (and this was the case early on for Anthropic). Instead it seems like releases are timed to build/maintain hype.
A thing to keep in mind is that if they release a smaller model halfway between well spaced big model releases, why wait so long on the next big model release if it's sufficiently ready to distill to a smaller model? The ability to demonstrate AI superiority is worth a ton, there's no reason to hold back.
The big AI labs are also accumulating huge datasets of expert work in a wide range of fields, which is very expensive to re-create. It seems pretty plausible that this this gives them a big advantage that is compounded by their larger training runs and larger models.
This is a differentiator, definitely, however I'm honestly not sure if that materially improves intelligence vs one-shot capability
There is not much open source AI .. there is open weight .. but anyways. Deepseek v4 is pretty much at the same level as the agents we had last year around November and it is an open weight model so I am hopeful.
I agree but we are dependent mostly on Chinese models at this point to pull it off.
where they came from doesn't matter so much so long as they are open weights and I can run on servers I control.
The final weights in themselves don't tell you anything about what went into the training process, e.g censorship.
Not today, may after the next 3-4 breakthroughs. One thing that people don't realize is that the AI they use today is highly highly subsidized bc of the capex that has gone into it. Even if people collaborated together - will not be able to raise billions of dollars that are needed.
These are still very very (and very) early days of the modern AI and there are so many changes that are gonna happen. It's possible that all the frontier labs of today won't exist in a few years.
I think we need it to prevent slavery happening again.
what is Open Source AI even?
to me Open Source, like Free Software, is something i can run on my own computer. any AI system that runs on a computer that i do not control is by my definition not Open Source.
so how then can Open Source AI win? it can't even compete. even if we collect enough money and create a dedicated Open Source organization to build and run a community owned AI datacenter, how does that help?
so what exactly is the demand here?
When kubernetes was released there were very few people who could run it, and even less that could run it usefully.
Right now there a few people who can run a 1T model at home, even less who can run a 5T model and probably single digits who can run a 10T model.
But if an open source 10T model was available you can be sure people would find new ways to quantize it, new ways to configure hardware and and new ways to think about problems that would make it useful.
1T+ models (Deepseek v4, Kimi K2.6 etc) are available as open weights now, and for ~$5000-$10000 you can run them usefully at home. 2 years ago no on was contemplating that.
$250K to run a 10T model might be possible now. There are many companies that will pay that, and that will push the tools and techniques downwards for the rest of us.
case in point: https://spark-arena.com/leaderboard
> any AI system that runs on a computer that i do not control is by my definition not Open Source.
This is not true at all. It would be open source if you could download it and run it anywhere that is capable, and are free to move it and modify it as much as you want.
Just because you don't have a computer at home powerful enough doesn't mean it isn't open source.
I think he means theoretically in possibility space, without relying on a based insider leaking a 'closed' frontier model to bittorrent or hyphanet.
Qwen models are actually very competitive with frontier models, and you can run them on your local computer. Gotta have a decent graphics card and by that time the current cost of the rig may not justify it over paying $100/month for cloud model but it’s all out there.
Qwen is still controlled by Alibaba, one company. We can't let the future be in the hands of a few companies, can we?
Fun fact: Qwen was not initially a Apache Licensed project, it was based on a custom license from Alibaba that restricts commercial use: https://github.com/QwenLM/Qwen/blob/ba2d85a13b28ed1ee0dde2d6.... There's no guarantee that they won't just switch it back later.
Kudos for them for switching to Apache License, of course. BUT, they're still a for-profit company. So as DeepSeek btw.
>Gotta have a decent graphics card and by that time the current cost of the rig may not justify it over paying $100/month for cloud model but it’s all out there.
Never, ever, subscribe. When you subscribe, they win. They cornered the silicon market to force you to subscribe. Don't be a sub, or at least keep your sub tendencies in the bedroom. ;^)
Please don't over-promise. -- An AI open³ dev.
Fluctuating token costs make it worth it
Projects like pluralis agora solve this problem. Really what you want is the model to be collectively owned and governed, not local
LLMs that you can run locally on hardware that is not out of range to acquire is already a thing for some time.
Recently I fired up Gemma4-26B-A4B on my 8-year-old PC... and it ran surprisingly well!
But I am going to need a much beefier machine to get it to the point where it can do any but very trivial dev tasks acceptably fast, and I'm going to need a much beefier model, perhaps one not so aggressively quantized, to keep it on task without the wheels completely falling off. Already we're talking serious money outlay, perhaps still within my programmer salary to accommodate, but just barely. And we're not even where near the performance characteristics a frontier model can support.
DGX Spark runs this sized model (I personally like qwen36moe better than gemma4moe) at speeds fast enough for interactive coding sessions. Algorithmic advances like DiffusionGemma ~4x token gen speeds (https://deepmind.google/models/gemma/diffusiongemma/)
We can run open weight models on our own machines.
yes, but a model that runs on my own machine will never have the capacity of a model that runs in a datacenter. as i said, it can't compete with that.
If RAM prices ever come down, you can have a machine that can run a capable local model.
Qwen 2.5 72B is surprisingly capable, almost on par with GPT-4o if not a little better. You can run it on a 128GB Mac Studio with 8-bit quantization. You need about 77GB for the weights and ~15GB for your context window & cache.
Pricing remains to be seen, but there's also those new nvidia laptops coming out the surface laptop ultra should have 128GB RAM w/ Blackwell GPU, they're saying 1 petaflop of AI compute, if you can tolerate Windows (no idea if it'll boot Linux until the hardware is out).
These models are roughly ~1 year or less behind the frontier models. We really just need hardware to catch up and alleviate the price pressure on RAM.
>If RAM prices ever come down
Maybe an unpopular opinion here (seening how Y-combinator is his baby), but I think OpenAI and Sam Altman should be financially decimated for cornering the DRAM market. What he's done is a step or two removed from what the Hunt brothers did. His buy-up of future DRAM silicon has measurably harmed personal computing, and he should not get to walk away with a 'win' from it.
> a model that runs on my own machine will never have the capacity of a model that runs in a datacenter.
I don’t think so. A local run model only needs to serve one or a few people. It seems possible to run a DeepSeek v4 model at full capacity on a server costing 200k usd. Very expensive but not impossible.
Factor in hardware and software improvements over time, and the fact that most people may just need to run a smaller and quantized model, it should take a pc at 10k usd scale.
It also will not change arbitrarily. Different strokes for different folks.
Huh? Open source is a quality of the software, not specific to the hardware used to run the model. The demand is that model weights are openly available for anyone to run and fine tune without restriction. Has nothing to do with the hardware it runs on.
Call it open weights if you must. But even with OSS just because you have the source code doesn't mean your machine is high performance enough to run it usefully this has always been true.
That was sama's and elon's original goals before they became trillionaires. Just to keep google/deepmind to take over.
Turned out both assumptions were wrong. You couldn't trust sama to turn this into open source, the Chinese did. Elon never.
And we couldn't see demis take over as expected, probably blocked by Google buerocracy.
For open source AI, AI infrastructure must be public just like roads, rail, ...
I think it's also important and heavily overlooked to develop and maintain open source "pro" level models. Those that are able to think for 80 minutes and yield heavy solutions.
I'm not an expert in LLMs so it's hard to understand how much are we lacking, is it just the compute and thinking strategies / parallel chains, or something specific architecturally. But I feel there's value there and I haven't seen anything like it available so far.
This should be the top post. Not Anthropic or OpenAI marketing plots. This is existential.
It's too late.
You can one-shot a port of Linux to Rust and stop contributing to open source.
The value of software is going to tend towards zero. The value of the software developer the same.
Anthropic is now a kingmaker. It gets to decide which businesses get the expensive private model that can generate entire business functions at the drop of a hat. If you can't afford the price tag, then competition in the market is not for you.
Computing is no longer "personal". It's for big biz only.
> You can one-shot a port of Linux to Rust and stop contributing to open source.
Touch grass brother. Seriously.
GP is exaggerating but I am convinced this will happen sooner rather than later. The improvements in AI are truly exponential if you read the SOTA papers. It's hard to keep up week to week.
It will win - in the sense that AI too will become a freely available resource. You can't stop progress.
My bet is that once cost-efficiency becomes a priority, we will figure out ways to get away from the expensive GPU infrastructure on figure out how to architect models for CPUs. I still remember that Microsoft paper about ternary weights.
There is nothing more surreal in AI chat than entering your own name and being told you are a banned topic. Open source models must win. There is no alternative.
My grim view is that it's just one incident away from some evil freaks to use ablated offline model for some nasty acts to have lawmakers lose their mind and try to regulate open source models and even consumer GPU. Think the latest 3d printers restriction.
> some evil freaks to use ablated offline model for some nasty acts
If this is a serious concern, why hasn't some red teaming effort demonstrated this possibility already? The fact of the matter is that ablation can't give a model world knowledge it doesn't have as part of training, it can only make the model confabulate. The "nasty" areas of concern are most notable for their world-knowledge requirements, which is where local models are at their weakest anyway.
> why hasn't some red teaming effort demonstrated this possibility already?
I'm sure they have but as usual we are a reactive society than proactive. Only when incident has occurred then we have momentum to act.
Don't worry, open source AI will win. There's a reason everybody is desperate to IPO fast and get an exit, their competitive advantage is not lasting long.
This is really a feel good argument and I agree with what he’s saying in principle but it offers zero in terms of a practical strategy or stable state where this is feasible. If you want to jump on the bandwagon then let’s put our pants on and offer a concrete suggestion that is practical and coherent. Otherwise what are we doing. Does anyone have suggestions to that effect?
What’s the world in which frontier model performance is open source? What does that look like? What’s a sensible business model that makes this sustainable? What’s a sensible regulatory framework that doesn’t hamstring AI progress?
Everyone is so enamored with these Chinese lab models like deepseek and qwen and GLM but they exist in a world where the top performance is still claimed by closed source models. These are not developed out of any benevolent commitment to the principles laid out in this article. A world in which OSS is the frontier and its development is controlled and funded by government subsidies of an autocratic government is not reassuring. You can inspect weights but good luck getting the cat back in the bag in terms of capabilities, safeguards, value system, bias, nerfing if it smells American business use cases.
Deepseek was such a darling but guess what, it’s now raising money — 300M at 10 billion valuation. OSS development isn’t sustainable as a business model and in a world where it costs a few hundred million to develop a frontier model, you need a strong business model, or you need strong state subsidies and incentives which introduce a billion new problems.
the most sensible economic picture of OSS models already exist. Commoditize your complement, passion projects for a hedge fund. These are unsustainable and exist at the pleasure of the business or the founder.
I am really curious how long will it take for the open source models to hit current fable/mythos capabilities, KIMI 2.7 was launched recently and its quiet good for open source models its as good as Opus 4.6 maybe in practical applications not benchmarks so like 6 months to an year behind, after which the next step will be to wait for the day when we will be able to run mythos level intelligence on local hardware, Remember when 5MB storage was the size of a table?
A loooooot of work to be done for the above to happen
There are two parts to this too. One is the raw model capability and the other is how well the harness guides the model and meets its expectations. I really think for stuff like agentic coding, this has to be treated as a package. This is my favorite example of how much difference a harness can make even for a tiny model https://github.com/itigges22/ATLAS
And you're bang on with the storage comparison, we're basically in the mainframe era of this tech, but there's no reason to think that it won't get optimized to the point where you can run the equivalent of current frontier locally.
The repo is so cool!
It's really neat project, gives me a lot of hope that we'll be able to run local models to do most of the work in the near future.
The win I'd really like to see would be for remuneration of training data, and for a provenance of all the data used by a given LLM.
I'm assuming this is popular because of Fable restrictions. AFAIK, open source is not excluded from ITAR / EAR restrictions (or other export restriction in other countries).
So the real solution you're looking for is technology that can't be arbitrarily gatekept by a sovereign nation.
I’ve been exceptionally displeased with Claude Code since end of February and switched completely to Codex in April. The blasé way in which one person (Borris) capriciously changes the system prompt multiple times a day, also no longer writing his own prompts (whatever that means).
That, the 5 different secret levers you have to pull to make it not stupid, the fact you hs e to go to the guy’s twitter account to find all the un-dumbing features and flags that aren’t documented anywhere else. That they decrease thinking budgets silently when they run out of compute instead of announcing the rationing, and gaslighting users at every step of discovery. The fact that internally they have their own coding harness and don’t use Claude Code primarily. The lack of formal evals and consideration for millions of users collective hundreds of millions of hours of investment in their workflows — that’s all off the top of my head, let me tell you how I really feel about what they did to Claude Code..
I adore gpt5.5 and maintain my own codex fork - but I have no idea how long I’ll get this performance / cost - I know it won’t be forever. I’d like to know precisely how much it’ll cost in hardware to run a gpt5.5 open source model locally. Hell a lifetime license to a model I can run locally is also be open to.
But I like building my own tools, from software to physical shop tools. I like being able to rely on my tools.
More responding here to the assertion that this is blowing up due to Fable.
I have been working on this exact problem, and I suppose now is as good a time as any to talk about it.
To make any agent "good", there are two components: the model and the harness. Very few companies can train models, but anyone can build a harness. How much does the harness matter? Can I build a harness that's good enough that I can use open source models with opus level performance? That's the question I've been trying to answer by building better harnesses. None of the existing frameworks have the functionality I need to build a good harness. The features I need are language-level... and so I started building a language called Agency[0].
It's been six months and its going well. Some of the things Agency can do are wild:
- It can pause and serialize execution at any point, making HITL easy
- It has some neat safety capabilities such as handlers[1] and PFA[2]
- You can bundle up any agent as an HTTP or MCP server[3]
- I'm now working on a built-in optimizer to optimize agents (think DSPy).
Obviously, it's a huge undertaking, but having worked with the Agency for six months, I can't imagine going back to another framework. It makes things so easy. I'm working on its built-in agent now [4]. My goal it to get it to be as good as Claude Code, but using open source models. It's still early days, lots of rough edges, but if this sort of thing interests you, I'd love to have a few more people test it out.
[0] https://agency-lang.com
[1] https://agency-lang.com/guide/handlers.html
[2] https://agency-lang.com/guide/partial-application.html
[3] https://agency-lang.com/cli/serve.html
[4] https://github.com/egonSchiele/agency-lang/blob/main/package...
I feel like this is similar to saying "open source cloud platforms must win". I'm not really sure what the concrete argument/proposal/strategy is here. Would open source AI be nice? Sure! Will the incentives of our capitalist economy change for this one specific product? Probably not!
It doesn't matter if open source models win or not. The bottleneck is the compute. When capital becomes cognition everyone other than the demigod class is cooked. We have a vanishingly small window to make sure that the benefits of large scale automation go to the species and not the owners. Once the owners become more powerful than governments or accumulate enough power to co-opt the governments we're done for. You can already see that creeping in along the edges.
While it is not at all practical to train an LLM with tens or hundreds of billions of parameters on hobbyists hardware, what if there are other architectures that perform just as well but are easier to train by 1000 volunteers?
I always wondered if 1000 1M parameter models fine-tuned to specific tasks with a small router could perform as well as 100B models.
And I know this is roughly how MoE works, but current MoE models still require training the model as a whole, and big players don’t have an incentive to change that.
But OpenSource community does…
It is practical, albeit not as efficient: https://arxiv.org/abs/2603.08163 . But organizing enough people with decent-enough GPUs is the challenge.
I feel with current government decision to block Fable, this is not a mere opensource issue, considering how US government restrict frontier models, what we need is sovereignty for every country. If not they will release every model with a kill switch in future like F35.
In the US -- once our nation finishes attacking our own education system -- this is definitely something a group of academic institutions could get together and accomplish. I assume the same is true in other countries. Companies like Nvidia and AMD might even support that effort, as they make money on the hardware and would probably be more than happy for there to be more reasons to use it. There may have not been a compelling enough motivation to achieve this before, but "models" didn't have this level of strategic relevance until relatively recently. Nvidia has been fairly good about releasing open weight models in the last few months.
Wait, which side is blocking kids fork taking algebra or forcing universities to admit people that can't do math or read, or abandoning phonetics for unproven methods that don't work?
stop talking please
https://www.usatoday.com/story/news/politics/2025/05/22/okla...
Both sides, since they are bought and paid for by the finance industrial complex.
It's the US, both "sides" of that coin are bad with examples pro and con all over the shop.
Still, to specifically give a partial answer to your poor faith rhetorical just askin' musing: Florida Conservatives
(specifically turfing nerds from New College of Florida and bringing an excess number of baseball sports bro's to a place that likes math and has no baseball field)
Open source AI will win. It's the same reason why out of all the languages on the web we could have used, Javascript won.
I think articles this light on content should not be upvoted to front page.
It's a perfect prompt for a rich HN discussion so while in general I agree with you, in this case the discussion is what matters.
This is almost always the case. Discussion quality went down during the last few years but HN is still _the_ place to attract people who really know what they are talking about.
I find that most arguments are endlessly rehashed. I would be like if most AI related discussion limited to maximum 2 / 3 most important news per day.
I think that the events of this evening (really of this past week) are almost unprecedented in the history of tech. Sometimes a clear and concise message is more important than nuanced analysis.
At d5s.tech we are recreating the layers built on top of models, working on dogfooding our own product to run a large chunk of the company.
I feel extremely strongly that a future in which most companies depend on one or two large AI-megacorps is going to lead to excessive rent seeking sooner or later.
I remain positive that the long term steady state will consist of proprietary models, -but- with open source AI models statistically close.
If compute keeps growing the relative cost of training current frontier models will decrease. An open source Fable/Mythos model simply seems inevitable.
Isn't training material the biggest problem for truly open source LLMs (such that could compete with top tier models)? The computation part can be solved with money, but compiling a comprehensive training set that could be freely shared and free of copyright issues is pretty much impossible.
Didn't the courts decide that if it's just for learning everything is fair game?
I wonder if we could gamify and democratise it somehow, like fold-at-home and wikipedia...
I've been training a teeny specialised model to run in a browser on a phone to detect harmonium notes played in a song (harmonium turns out is a pita, another story for another day), getting good labelled data is _all_ of the hard work.
That being said, maybe for cheap inference, using a big model to train something ultra-suited for the task at hand might be how we could handle local inference; thinking language specific models.
You don't need to have fully copyright-unencumbered datasets to build Open Source AI, as that (as you say) would be impossible. https://opensource.org/ai
Truly, big corps have no incentive to invest in open source local AI. I maintain a small effort towards this goal here at https://pocketweb.tools/
The article doesn't say what it means by win. I presume we will have the present situation where the cutting edge stuff is closed source developed by profit oriented companies and open source is available two but a year or two behind.
i think to create or make opensource ai need competition power and alot of investment to create and use or use it local you need spec pc to run and tune it at minimum 27b model to act good on context and agent work
I would also want all conversation with AI to be public, searchable and indexable.
It is only fair, give that LLMs are enabled by human generated content from the Internet, that they give it back!
I think it's enough to use Open Router to encourage competition in the market place.
The latest US gov meddling in the Fable rollout really put the nail in the coffin. We can't integrate a strategic product that is subject to the capricious behavior of the US
As an person whos getting into tech and already developing a game, the fact that laptop prices since 2020 have increased by 20-40% is insane. It's delaying the time to create my game. I researched the reason for the cost spike, and most of it is from the excessive money put in ai Technically, the owners of AI could slow down the amount of GPUs and RAM they buy because AI has almost reached its most usable peak. Everything they add just introduces more bugs, so instead of building more AI centers, they should focus on improving the main AI model with bug fixes. There's no need to give it more unnecessary power. Most people don't care; the entire business is run by a few old men who think AI is everything and invest huge sums of money to show other AI companies they need to improve to get more funding from old people. We just need to find something new and innovative for older investors to focus on, so not everything is about investing in AI like Roblox, OpenAI, Google, etc. The extreme amount of reasoning power given to AI is causing bugs, and the moments when AI had outbursts towards people are related to this.
They want to corner the compute market and destroy the personal [sovereign] aspect, so that you are forced to subscribe and pay them regularly [indefinitely] and the US security state can surveil you. Never subscribe, and never buy products from companies who subscribe. Starve them, bankrupt them! We do it by not subscribing!
> because AI has almost reached its most usable peak
It doesn't seem to be showing any signs of stopping. Have you used Fable 5? It's a fantastically capable model and trumps anything that came before it. Seedance 2.0 is categorically the best video model, and it's only a few months old.
> the entire business is run by a few old men
Startups tend to skew young, and in this case it's no different. Most of the leaders of AI companies are decades younger than the CEOs in other types of industries.
> who think AI is everything and invest huge sums of money to show other AI companies they need to improve to get more funding from old people.
They're spending capital to win market share and to try to build a moat. One of the most important things in business is building a durable way to keep competitors from taking your market. You spend enormous capital to win customers, and it would suck if other businesses could watch what you did, spend less money, and come in and take everything away. The money being spent is an attempt to have a durable lead.
It's working. Enterprise contracts are deep and sticky tendrils that work through governments and large companies. Both OpenAI and Anthropic have massive partnerships with Fortune 500s, the DoD, you name it - and these contracts will last and print enormous amounts of money. This makes it incredibly hard for other players to enter the market and build a cash flow with which to compete and thrive.
> find something new and innovative
This is easier said than done. It's an incredibly hard problem. It took decades to find the last big technological waves: the PC, the internet, broadband, smartphones. Now AI. These are generational step function increases. The groundwork can be decades old, but it takes time to proliferate before it can become a big business.
Other possibilities include fusion, green tech, quantum computing (useful for crypto breaking, etc.), AI drug discovery, etc. If you go into research one day, try to find an interesting field with potential for commercialization - that could make you very wealthy if you find something you enjoy working on, with lots of greenfield opportunity, that is ripe for turning into products.
Good luck with your game! You should post it here on HN when you finish. You'll get lots of great reviews, comments, and early players. :)
thx I will consider what you sent.
Why have you sent this same message multiple times?
I didn't know how this worked I thought it deleated it, at first.
I think models will be a commodity sooner rather than later. This whole race doesnt matter. First mover advantage is real, but over enough time it wont matter.
we could've been fine with the sole existence of AI if the organizations providing them weren't greedy and rug-pullers. anthropic could've been loved by all if it acted towards the benefit of humanity. as intelligent system continue to become smarter, close or beyond mythos level, what now? with the 'community-driven' mindset we have, is the future really going to be safe? probably not we just need a company that develops, serves, maintains, these models the right way, priced fairly that benefits the user and the company.
Well, the crazy thing I'm working on (100% self-funded thus far): https://trivyn.io. The main idea is moving most of the reasoning to the symbolic layer so the "neuro" piece can be a small model able to be self-hosted on reasonable hardware.
If open source AI was better than what it is currently chasing, wouldn’t that take away the incentive for these companies to give it away for free? Training is expensive and companies will need to recoup those development costs once it stops being about jockeying for position.
And it will, but be patient. I took linux 25 years to conquer the world.
One day an open source model reaches "good enough" level. Maybe around the level the current frontier has and most people will use that
I don't even need today's frontier, give me a local model I can run on my Mac comparable to Claude 4.5 as of December last year and I'll probably lose any interest in new hosted LLM advancements altogether.
If you've been writing off Deepseek V4 Pro, now is your time to go set up moonbridge and give it a shake. It's exceptionally good.
Got a bit more than 1B tokens for $10, it's exceptionally fast, it was able to fix/implement things that 5.5 xhigh struggled with, without trying to act like my best friend or do that coy "undersell the ideal end result so that it can later overshoot it and claim a great success" bullshit.
E: miss me with the "but China" BS, everything I've experienced while using this model has convinced me they are earnestly more concerned with doing the right thing than Anthropic could ever pretend to be. And if you want to ask it questions about Mao, you can go download the weights and spend mid-five-figures to fine tune that out.
For US citizens: counting on Open Source AI is another libertarian fantasy.
Open source AI should and will get better for sure (including better defined first), but the state will have the power over AI never the less.
If you don't like govt's AI policy or the people making those policies, go fix that, don't act like you can avoid them.
For Chinese: saying "Open source AI must win" sounds like singing "L'Internationale, sera le genre humain". The reality is Open Source AI will be over the moment US competitive pressure gone.
For rest of world: there's no real AI for you so far, go work on it or be a citizen of US&A or China.
Well, open source AI is mostly coming from China. Title should have been China must win.
Civilization is at a crossroads, or will be soon. Democratization of AI can be good up to a point, but existential threats can also be real, and democratization of existential threats is not a survivable policy.
It's actually the opposite. Democratization of intelligence is the only way to stop existential threats and render them useless.
Right now, and likely forever, because biological threats can be sanctioned at a supply-chain level, the risk of AI is all digital. Fraud, phishing scams, spam, hacks, etc.
The only way we harden the worlds infrastructure to the point that it can withstand attack from bad AI is if we have an abundance of access to frontier intelligence to develop countermeasures.
Otherwise, bad actors will develop these capabilities behind closed doors and use them to hold the world hostage and cause irreparable harm. There's no putting the genie back in the bottle. Good and open-access AI and the people using it are the digital immune system.
If there's an asymmetry where bleeding edge is gated off to only a small group, and allowed to gain exponential power over the immune systems defense grid, the slightest infection will lead to death of the host.
That's a thesis.
Available components must win. I’ve often been a critic of open weights and open architectures that give very few normal people access. What’s the point of releasing the plans for a nuclear reactor if no one can have the fuel?
I vote that you become the next Richard Stallman
…just crowdsource your own data centre…
what if grok went open source and was on par with open chinese models? the business play may not be the models themselves but owning the data centers and running infrastructure for all models from all companies? a lot of people could then be rooting for xai and elon could ironically save face by actually implementing an open model
You can do that now. There are many different providers for Deepseek already.
I hope so. But how? Who gonna fund these projects and how to coordinate with every sides. This is complex. I only believe that the open source AI won’t lack users.
I'm already coding more with DeepSeek than Opus, I'm doing my part :)
even if the most powerful ai is open source and let's even assume runs on consumer hardware - in the end data is the real moat.
if it can access private data it will necessarily have more power.
Let's abstract this further: It's about the user's existing power and intentions, meaning if I am already in a position of power, AI will multiply it to levels way beyond a peasant could. Power dynamics just get exacerbated.
If we can't stop these big AI companies, we must to put force that everybody can see what they are hiding from us.
Not to distract from the message, but I appreciate that this is largely plaintext not React vibeslop.
Winning is a tall order. I'm just hoping it'll get good enough while allowing us to run it locally with no idiotic "safety" controls or censorship of any sort. Looks like the best open weight models are at Sonnet level, if they get to Opus 4.6 level it's gonna be perfect.
Replace America with „The World“
What does the author mean by "win"?
Does he mean that the _best model_ should be an open source one (eg: today, something better than Fable 5), or just that open source models should be the default choice for most task?
The former seems an impossibility, closed labs can work off of open and their own closed research. Closed source will always be better. Well, at least until some late-stage enshittification dynamics cause the providers to hobble them.
The latter, becoming a default, not so much. But considering the deep-rooted nature of (for instance) Google, it certainly won't be a walk in the park. This seems to be a similar hurdle as dethroning Chrome as the default browser.
For the average ChatGPT user, I surmise that open-source models are already capable enough. Most people I know who use it (me included) are not paying for it, they are routed to the cheaper models.
What's needed here is everything else other than the model to be in place. Which is to say there isn't a sufficiently good open source ChatGPT app, every open source option requires more fiddling than the ChatGPT app.
No precedent comes to mind for non-tech-user software that is open source and also a default choice. The limitation is rarely from the core-tech capability; core-tech is often the same as what closed source uses.
Are there any platforms to discuss this? (Like matrix/zulip?)
This is obviously in direct opposition to the very idea of America. So if it happens it'll happen in other countries.
I’m sorry I can’t read past the first paragraph
Definitely, but I see the gap widening everyday, especially while commercial AI models have started converging towards AGI. However I do believe and support the cause, as it's the next big thing as developers we need to take to prevent a complete monopoly in the coming few years.
"Converging towards AGI"
These things can't even center a div correctly half the time.
Not everything is code. Just because it generates a shitty SaaS clone for you and that seemed magical, it does not mean we are approaching "AGI".
An AGI could design an Oil tanker, manage the project from start to finish, handle all contract negotiations and purchasables, payroll, scheduling. Then it could do that 50x over and start a leading logistics firms.
In reality an LLM can't even complete upwork projects that are worth $20 an hour more than 4% or the time.
Source:
https://labs.scale.com/leaderboard/rli
4% guys, 4%. It cannot complete entry level work on fucking Upwork 96% of the time. Stop falling for the marketing and sorry but an LLM will never be AGI.
Its literally just text autocomplete with some RLHF post training, holy shit im losing my mind. I want this hype to end so badly holy shit I need this to end.
i guess this fits: https://thealliance.ai/projects/tapestry
It sucks how in just a few years the world has decided nothing is worth doing or is just impossible without the use of AI. As if regular human intelligence isn’t enough anymore and it has to be paid for somehow.
This should definitely attract a lot of contributors
There is no such thing as "Open Source AI". Open Source means that you respect copyright. The types of AI models that this web site refers to do not. Stop this nonsense!
To me it does not matter whether AI is open source or not. Yes, it is better if it is open source, no doubt, but either way I think AI must die. Naturally it won't, we all know that, but this does not change my statement in the slightest - AI must die. Having it open source is, while an improvement, just painting lipstick on the pig.
What about pirated models?
There are no open source LLMs.
If you take AI risk seriously then Open Source AI should not and must not win. Both by evil actors (biological weapons research) and the danger of unaligned AGI itself. There are some people who would never work for the military or Anduril (automatic weapon systems), but an OS AI „without asking permission“ would be the same.
If closed-source AGI wins, it is not going to be much different from a safety perspective anyway, because AI capability research is advancing faster than safety research.
Closed Source AI at least can be controlled. See the directive of the US government regarding Fable (even if one disagrees about the directive there is no doubt that it is effective in shutting it off) or the safe guards by a corporate structure (even a profit driven one). It is schizophrenic to praise Anthropic for refusing the Department of War full access to their models but at the same time root for Open Source models.
Edit: relevant Scott Alexander article from today
https://www.astralcodexten.com/p/my-ai-opinions
> In terms of bioweapons, I expect that closed-source AIs will be heavily optimized against helping with these, and open-source AI will be banned after the first warning shot (or become economically prohibitive even before then).
Note that warning shot in that blog post means specifically a near-disaster event (perhaps one that's just barely averted) that's specifically caused by the AI. So far we've had AIUI no significant indication of open-weight AIs being problematic in that sense, whereas one can quibble that proprietary AIs have done dumb and dangerous things.
(For example, I suspect that plenty of folks would view the recently threatened mass scan of the DN42 hobby network as an instance of misaligned agentic behavior that would have wasted non-trivial resources, and I also think that most observers would pin the specific behavior of that AI on a proprietary SOTA model, not an open one. That's clearly not a disaster-level event, but it should scare you if you're concerned about alignment.)
it is inevitable that it will win
information wants to be free
This is not about information but about capital. Even if we had free access to the weights of the best models in the world: who would be able to run them?
Technology is deflationary. I am holding in my hand a device that would have been a supercomputer 30 years ago. It costed me a couple of hundreds of dollars.
These models and the hardware they are running on will get even more efficient. We are nowhere near the physical limits of what we can achieve.
> Technology is deflationary.
Not anymore! Well, if you're like Elon and already taking down the bottle of Cuatro Comas from the high shelf, the economies of scale will continue to work in your favor.
But one of the really neat things about AI is that there is no limit in sight to the scaling incentive. More compute will always get you more: more training, more inference, more parameters, more capacity to build more and better models, more spare capacity to run the slop your models have already built to generate the slop that will succeed it. Back in the dot-com days, or even the "big data" days, you wanted to scale up rapidly but there was a limit: there were only so many customers and they could only produce so much data you could only ingest so fast. In the late 90s, one of the world's most trafficked sites, ftp.cdrom.com, ran on a (single!) dual-processor Pentium Pro system. That was just serving files, and there was certainly room for more CPU oomph to provide more sophisticated services to a huge customer base. But once those customers were served, more compute, storage, and network capacity didn't buy you enough to justify the capex. That is emphatically not the case with AI, and so the incentives for the AI companies are to buy as much compute as they possibly can. What this means in practicing is pre-purchasing capacity at the semiconductor fabs to manufacture chips exclusively for you, and there's only so much of that capacity in the world. Trillion-dollar companies can easily outbid the entire consumer market, and so the incentives for the fabs are now to sell to AI companies at the expense of the consumer market. That's why you're seeing memory prices go through the roof. Modularized RAM for end-user PC builds will soon go the way of the CRT: it will cease to exist as a market product, it won't be manufactured anywhere by anyone. GPUs, CPUs, and storage will soon follow. The only devices end users will be permitted to purchase are all-in-one integrated devices, with CPU, RAM, GPU, storage, and networking either integrated in-chip or soldered on, and they will have just enough capacity to connect to the cloud services the user wants most to use. Most likely, you will be permitted a subscription to such a device, with automatic hardware upgrades at periodic intervals supplied by the manufacturer. If your subscription lapses the device bricks itself. Almost certainly, the OS will be locked down, with no end-user option to install a different one or even run unapproved software.
If reasonably powerful computer hardware for end users exists in this future, it will be available from a single company: Apple. Only they have the leverage to prevent ~100% of manufacturing capacity from going to high-roller, big-tech firms.
> Trillion-dollar companies can easily outbid the entire consumer market
I don't think this is true. I think prices are rising at the consumer and prosumer level because that's what's required for the mass market to collectively outbid the handful of trillion-dollar companies, at least for the limited share of production they can sustainably demand. This process can continue pretty much indefinitely.
> But one of the really neat things about AI is that there is no limit in sight to the scaling incentive.
How you can be so confident? I can imagine there is some limit and with each scaling iteration gain you achieved will decrease so that further iterations would be more and more look pointless
I'm sure a limit will come around eventually. But plans are afoot to build city-sized data centers, and even then that's not enough to sate the AI superscalers' ambitions, hence Elon's talk about putting data centers in space. This is a level of compute scaling unheard of in our lifetime, and we're still a long, long way off from AGI. So while the juice may theoretically not be worth the squeeze at some point, with the current capacity we have there is no end within sight to the incentive to build more. It will take a number of years at least, and who knows how much environmental/economic destruction, before the dropoff in return on capex begins in earnest.
Well it would be anyone that has access to a datacenter to run them. Which is a ton of companies. And those companies will rent out access to those models. And if they do something stupid to screw over consumers, well the whole point is that there would be a bunch of companies that you could use instead.
Inevitable isn't "in our lifetimes"
We've never seen open source win before so I'd be dubious that it can win here without concerted effort.
Every machine nowadays runs Linux in some form and Postgres is the default database.
Indeed. And are most of those machines being used to run open source applications? No.
“information wants to be free” - doesn’t seem correct. More like it’s easier to spread info than to hide it.
Intelligence is now data in the form of weights.
And once it leaks, it's permanently in the wild.
Interesting times.
"intelligence"
K
I fully support this. How can I help?
Open source ai will win.
Anthropic just kneecapped themselves, and possibly OpenAI and Google as well, with their FUD strategy that got fable shutdown by the government.
But that doesn't impact Chinese providers. Then can US companies get investments for expensive model development if they can't actually sell those models-as-a-service?
In the meantime, open source will continue its march onward because while slower, it's completely open source, and the models are already good enough to improve their own work as well as build out the next gen of models.
Open source AI wins means China wins
Did open source phones win? No, iPhone is pretty dominant.
Did open source operating systems win? No, MacOS/Windows are pretty dominant.
Does open source... cloud hosting, social media, ride sharing apps, you name it win? Not in my experience?
So I've long said that the valuation of OpenAI at a trillion(ish) dollars depends on OpenAI "winning" and "owning" AI and there being a sufficient moat to stay ahead of competition. Without that, the company is worth a fraction of that. Anthropic is probably positioned better here actually but it's still kinda true there too.
Ever since a Chinese firm released DeepSeek I immediately came to the realization that any US tech firm "owning" AI is simply not going to happen. China will make sure of it. It's in their national security interest not to let that happen.
From the POV of geopolitics, IMHO the US shot itself in the foot by banning the export of the best chips to China. The US also somehow has the power to prevent a Dutch company (ASML) from selling to China too. That makes a little more sense to ban but the combination of banning EUV exports AND banning the best chips sowed the seeds for the destruction of all of this.
By banning chip sales, the US inadvertently created a captive market for Chinese chips with Chinese companies. If there were no chip ban, Chinese companies probably would've bought US chips. But they can't. So they can only buy from Huawei and SMEE (indirectly). The US forced China to realize it was in their national security interest to copy the best lithography and, by extension, the best AI chips.
So DeepSeek was reportedly developed on either older NVidia hardware or smuggled newer NVidia hardware but that won't last either. At some point it'll be completely Chinese made chips that are doing this.
And what's the biggest cost for a model? Training. But you do that once and the model like any software is infinitely copyable so China can under OpenAI, Anthropic and SpaceX (xAI) and that's what they're doing.
But it gets worse for the AI moat. Local models are going to get cheaper and cheaper to run. You can already run 31B models on sub-$5000 hardware. What do you think it'll cost in 5 years? Will larager parameter models keep getting better or will there be a law of diminishing returns? What is a B100 workload now, will be a Macbook Pro workload in as little as 5 years.
What if all these AI data centers are ultimately just going to be commoditized cloud hardware like AWS in the not too distant future? We already see Google renting big from SpaceX. I think the writedown on all these data center investments and the companies that are doing them is going to be extreme in the next 5 years.
Latest deepseek was trained with Huawei chips I think that's why the development velocity was rather slow from V3.2 onwards.
Unfortunately General Secretary Xi isn't as AGI pilled as Amodei.
Never thought I would say these words, but:
Good Guy China! :DDD
our dependency on US AI will lead to data concentration in hands of few megacorps.
I hope the news moves this debate past "open weights vs. closed APIs" as the only axis. Open weights matter, definitely, but applied AI also needs open infrastructure around the model and it feels a bit like I'm yelling into the abyss highlighting the future we're incentivizing - cognition rented from a few institutions with access changing based on policy, geopolitics and platform incentives like advertising
Totally agreed!
I mean, even if the frontier labs opened their frontier models, only nation-state level actors are capable of running them. A lot of the tech is very open and known, its putting it all together that's the struggle.
If any AI wins, how can that be good for humans? It's high minded but if any AI wins, why would any of "The ability to study, build, repair, deploy, audit, adapt, teach, preserve" be important? Is the real problem to be solved something else, if you want those things?
It can't.
Hear me out, economies of scale can only be met when there is a large enough liquidity for it.
The amount of people willing to purchase multiple hardware releases year after year just to run LLM is already tiny and businesses already do use their own hardware and there is no desire for manufacturer to reduce their own margins.
This will never work - a strong enough LLM model will also let you synthesise bioweapons etc.
How can you release this to public?!
Why else do you think Anthropic is heavily restricting Fable? You can’t just handwave safety concerns.
the public only wins once we shut it down globally through treaties like other tech that's too dangerous for anyone to have
it is baffling that you can still encounter Yuddite delulu in 2026 when everyone and their literal grandma is using chatbots daily. you might as well campaign to shut down the internet or ban smartphones.
but ok, who is going to initiate such a treaty? US? the orange man won't, and even if he did, no one would care. by the time his term is over and the next AIPAC spokesperson is elected, it will be even more late than it is now. EU? impotent and irrelevant. China? lmao.
Given that it's most public use in open source so far is to whitewash GPL code into MIT code, no, I'm sorry, I don't think "open source AI" is particularly important.
It’s the GPUs, not the weights that are the key.
As long as these models require a lot of computing power, the best models open source or not will be served by corporations who can afford the infra.
It likely won’t based on how SOTA are developed.
But if "they" stay on the current trajectory we'll never own hardware capable enough to run the open source AI. They want us to rent everything from the cloud and never own it. If a government-supported cartel forms around this idea (which appears to be the case) that's the end of it.
Open source projects are only successful when they make what they replace obsolete. This worked with Linux and GCC but this isn't gonna work with LLM's.
Who's gonna pay to power an open source AI? Will it perform well enough to make Chat-GPT and Claude obsolete?
Not to be that guy, but the correct term is Open Weight LLM. And I’d argue it already has. Many open models are already very competitive with closed models at a fraction of the cost.
Labs can and do open source more than the weights
https://allenai.org/olmo
In the end it will win in some universes and lose in others, just like the Nazis.
All we can do is hope we end up in the one where things are ok.
The only way for open source to win is for closed source to provide the compute resources.
That’s really the only thing stopping people from training or running these models at home:
This is why I go for codex and not claude code for example altough the models are not os yet.
ok just explain the cyber attack and bioweapons risks like we are 5. Didn't you think of that? come on
Were it not for China, America would have restricted the most advanced models from being used outside the US. NATO members would have access to GPT-4, with some countries entirely blocked from AI.
Biden's GPU controls should give you an idea. Thank you, China. Open source AI must win.
Unfortunately the US is no stranger to using export controls to restrict frontier technology.
Famously, the PowerMac G4 was briefly subject to export controls. Apple turned it into a marketing campaign.
Just happened 5 hours ago.
China unironically saved humanity. I'm no fan of the CCP but if they hadn't organized an effort to compete with the US no one else would have done it and we'd be begging our AI overlords for tokens and praying we don't get caught conducting wrongthink.
Go ask Claude to criticize Anthropic and see how long your account stays active.
Never rent. Never subscribe.
Subscribing is cuck paypig behavior.
You're not a cuck paypig now, are you?
Pass this on to your frens, it may save the future!
BAP BAP BAP goes the Billionaire Alignment Problem
Yeah except for all the money it costs to do well.
Quick, someone start open data center and open energy system and open water supply.
> If intelligence becomes something people can only rent from a few closed institutions, the public does not just lose software freedom. It loses operational freedom.
And people do not just lose operational freedom. They lose the freedom to think, much less act. To some extent, general intelligence has already been outsourced to a few companies. Phones and computers extend the human mind's capabilities, but most people don't have root on their phone. They don't know or control what software is running on it, or how the hardware is made. They don't control their phone, the phone controls them instead. The upstream problem is ownership of general computation, ownership of your own mind, aka self-ownership. This will become more obvious as computing devices become more personally integrated (desktop -> laptop -> smartphone -> smartglasses -> neural interface). Who owns the digital part of your mind? It's not really you at the moment.
Democracy, or any form of negotiation, can only exist among entities with similar capabilities. The gap must be very small. Orangutans may be smart enough to drive a golf cart, but there are no orangutan citizens in a human democracy. So you cannot run from this by being a luddite hermit in the mountains. When the world is full of digitally computing humans much smarter than you, you'll be at their mercy like monkeys are at the mercy of humans. We destroy their habitats and experiment on them as we please.
Now for the first time in history, organisms can increase their own information processing capability at will. We're in the middle of a speciation event where humanity splits into those who own the digital part of their mind vs those who don't, and there will be further splits based on how much compute you own. Though a future where no individual can fully own their mind is also possible.
By "own", I mean being able to command the entire technology stack. If we want sovereignty for the masses, then we must decentralize the entire technology stack for general computation. That means everything from electricity generation, to chip design and fabbing, to all layers of software from firmware to neural networks. All of it must be accessible to every individual. Everyone must be able to make a computer from scratch at home, or at least without leaving the city they live in. Anything less than that, and democratic society as we know it will continue to crumble.
The fundamental idea underlying all of this is: that which reproduces, survives.
At what level of organization can we reproduce?
The digitally computing human species cannot reproduce as individuals. We can only reproduce as a society, at least for now. You can't make a computer from scratch on your own, but you can make a brain from scratch with just one other person of the opposite sex. As the world we live in becomes more suitable for the digitally computing rather than the purely organic, the organic part of the digitally computing human becomes less likely to voluntarily reproduce. If the organic part were to survive without being disempowered in the future, then it's probably by moving the mechanisms for reproductive drive to the society level (via religion or authoritarian government incentivizing or mandating reproduction), or by ensuring that each and every individual has the means to make the digital part of their mind on their own just like how they can make the biological part on their own.
Wasn't it the point of ... OpenAI?
A website stating the obvious, given small target audience it will probably reach, and a call to arms consisting in emailing a random unknown person.
We're saved /s
Instead of doing a vanity site with a shelf-life of a few days, see where the action already is in online local LLM research and communities and contribute.
Isn't that OpenAI's mission? "Our mission is to ensure artificial general intelligence benefits all of humanity."
/s
Can we assume that the author isn't using "Opensource" to mean "Openweights"?
Or are we still collectively brainwashed by the strategic false equivalence established by Big AI CMOs?
On this very thread you already have people talking about "open weights" and similar nonsense. What is open about them? They're free to download, but that hardly qualifies as open. Where is the source? Where are the instructions to modify and build your own?
I'd never though I'd have to utter the expression "open as in beer".
The blatant attempt at manipulating vocabulary here is... quite blatant.
I'm a strong proponent of Open Source (TM) but I disagree with this take.
The weights are the useful artifact here. You can modify them, fine tune them and do what you want with them.
Unlike binary software there is nothing limiting that.
It is also useful to have access to the training recipes and to some extent the data. But I'm of the opinion that learning on something is not copyright infringement, so there are many circumstances where distributing the raw training data will not be possible.
For me this is like Open Office: it is open source, and largely inspired by and learned from Microsoft Office. But they don't need to distribute MS Office for Open Office to be Open Source.
In addition there are models that meet the criteria you appear to propose. The AllenAI models are a good example.
The analogy falls apart very quickly. Without the training data, your modifications amount to virtually nothing compared to what these "versions" are, and the idea that you can maintain and improve on these models without the continual support of the company that owns the training data AND harnesses AND in general build instructions is not very credible. This is why it's not rare that they "dump" old versions as freeware but at some point switch to not distributing them, and mostly get away with it. As this is really not open, and the threat of an effective fork is therefore non-existent, the pressure for any one who has released freeware models to "go SaaS" is too high.
While if "Open Office" switches to a more problematic license at some point, the existing source has all you need for an organization to support the project without regard to the original company (this has happened already!). If Qwen decides to stop distributing models for download, you're basically stuck, _even_ if you have unlimited resources, it's not clear how the released weights help you; your best bet is to start almost from scratch. This has also happened...
These models are not "Open" by any definition of the word. It is just freely redistributable. You can justify yourself in whatever way you want re a cowboy approach to copyright, but this doesn't change the fact that this is not open, and has almost none of the benefits of open, and therefore it is a huge abuse of the word "Open".
Ironically about the only thing that is copyrightable here is the sum of the training data (possibly) _AND_ the software used to build the model (most definitely). The model itself most likely isn't (databases are not copyrightable), which makes it even more pointless to abuse the word "open" for it. All the value is in the former two.
What would the 'source' be for an LLM? There is the structure, and the weights, there is no 'source'.
In case you're not just trolling, please learn how "the weights", which are analgous to a compiled executable, are made.
There is no source because it's not software. You can of course modify and make your own.
> If intelligence becomes something people can only rent from a few closed institutions
Just your your natural born intelligence..? It's worked for the past 10k+ years, I'm sure it will work for some time longer