Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time.
Are Anthropic and OpenAI rushing to IPO for immediate cash so they can delay the inevitable? Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end.
We are only just now getting a taste of the “true cost” of these tokens. Then there is a lack of compute bottlenecking everything. Even now I’m looking at the 7.5x rate of tokens for Opus 4.7
Open models are promising and cost a fraction of what they proprietary models cost which the big two are vulnerable to when companies start to feel the cost of tokens.
Will data centres be built fast enough and powered sufficiently to lower the cost of compute thus tokens?
Is it just a giant Hail Mary to get to AGI ASAP before the economy collapses?
Above all else, I simply feel the models have plateaued. I am noticing productivity loss for tasks I deem as “complex”
> Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end.
I think a LOT of companies really never needed to be on the public market, and its a darn shame that so many go on the stock market, we have this obnoxious culture where you have to fire tons of people if you have a bad quarter just to show you're stopping the bleeding. Companies literally fire and hire x number of people every quarter to keep things going, its ridiculous and unhealthy. Private companies rarely work like this, I'm sure there's exceptions.
Every company I've worked at started off private, and those were their golden years, until some economic hurdle happened so they sold it off to a bigger fish who is on the stock market, who bought them to be more attractive to investors or what have you.
I wish there were an alternative to the stock market where you invest for the long haul, and you cannot take your money out in x number of years. I think this would make more sense. Maybe it doesn't fix the VC peeps want their money back nonsense, but if you could do it even for early stage companies, maybe it could help somewhat.
There is nothing that stops you from buying stock and holding it forever, Buffett does this.
There are very stable companies in the stock market, like Cocacola. But they are not glamurous and don't give headlines.
And there are enormous fish in the private market, e.g. Cargill.
Stock markets are great if you have a company that needs money to expand quickly, and don't mind to share ownership. Stay away from IPO-jackpot stuff, and it shouldn't be that awful.
I think it's less about being able to buy and hold stocks, and more the effects that going has on an organisation because you're now beholden to shareholders who expect returns causing the decisions made by the business to prioritise short term gains
> I wish there were an alternative to the stock market where you invest for the long haul, and you cannot take your money out in x number of years.
That exists already! People often complain as well when a company ends its golden years because of some economic hurdle and ends up being acquired by a bigger fish who is _not_ on the stock market.
It's less about the company leaving the stock market and more about "Private Equity" often being a legalized embezzlement scam designed to suck the company dry and then dump its withered husk in bankruptcy court.
So you're asking for some type of equity that's private?
Seriously though, I have seen some very large companies like Tibco and Dell go private for an extended period of time as a means of avoiding shareholder nonsense during restructuring.
Its one of the reasons Valve is considered such a great company by its customers. If they were a public company, they would be enshittifying everything in an attempt to scrape every last penny they can.
They choose to do so because they've lost money in a bad quarter, which might not be the case on the next quarter, its ridiculous. I would rather invest in a market where my investment is long term based, and you aren't just firing people to make numbers work. To these people its all about make the numbers work for investors, they don't care about anything else because of the way that market works. You can offramp your investment on a whim, which is ridiculous and volatile at times. Personally I would prefer more companies go private. Some companies probably wouldn't exist without the public market, like some social media companies, and maybe that's okay if they did not...
Let companies fail, but also lets make investing smarter.
From the limited perspective of software development, today’s models are well-worth their per-token cost.
This reads to me like Anthropic anticipating demand and making a commitment to acquire supply. Not unlike airlines committing to future jet fuel purchases, or Apple committing to future DRAM volume.
> From the limited perspective of software development, today’s models are well-worth their per-token cost.
At the current price or real price? Anthropic said a $200 subscription can cost them $5000 so the real price could be anywhere from 10-30x the current price.
No, that is probably one of the worst cases they probably saw. Most likely the subscription inference cost is much lower than you expect. If you look at costs for similar open models they are much lower than what you get by buying from anthropic, so that is the real cost basis I expect.
It's likely Amazon is making a fucking killing though.
While $5000 is a lot, the people who rack up close or just over a thousand "API equivalent cost" are pretty common.
> Most likely the subscription inference cost is much lower than you expect.
This is probably not true because they'd be screaming it off every rooftop were that the case.
Same deal with the API inference. Even the "profitable on inference" claim is sourced back to hearsay of informal statements made by OpenAI/Anthropic staff. No formal announcements, nothing remotely of the "You can trust what I'm saying, because if I'm lying the SEC will have my head" sort.
Yet making such statements would be invaluable. If Anthropic can demonstrate profitability before OpenAI, they could poach most of the funding. There's no reason to keep it a company secret.
And API inference is only part of the total costs, not even bringing in training and ongoing fine-tuning. If they're not even profitable on inference, how could they hope to be profitable overall.
I'm going to be a dickhead for a moment here, apologies, there's no way to say this that isn't rude to you. This is still the same hearsay "In an interview, somewhere."
> Let’s say half of your compute is for training and half of your compute is for inference. The inference has some gross margin that’s more than 50%.
But the context, the very previous sentence is:
> Think about it this way. Again, these are stylized facts. These numbers are not exact. I’m just trying to make a toy model here.
Here, Amodei is in effect using weasel words. He is not giving any actionable claims about Anthropics margins, merely plucking an arbitrary number. Why 50%? Is 50% reasonable? Is 50% accurate to the company? Those are all conclusions the listener draws, not Amodei.
> I don't know about SEC rules
The main premise is that, as a CEO, there are some regulations you are beholden to. You're not allowed to announce you've made a trillion dollar profit, sell all your stock, and then go "teehee just kidding". The SEC prosecute you for securities fraud if you do that stuff.
This makes such weasel words as earlier suspicious. Because the exact statement Amodei gives is not prosecutable. He's not saying anything about the company, just doing a little "toy model".
The degree to which it is intentional that this hearsay travels and is extrapolated from "Well he picked 50% because it's a reasonable figure, and because he's CEO, a reasonable figure would have to be a figure akin to what his company can achieve" into "Anthropic has 50% margin", that's up for debate. Maybe it is intentional, maybe Amodei is exactly the same kind of shitweasel as Altman is. Probably he's just a dumbass who runs his mouth in interviews and for whatever reason cannot issue the true number in an authoritative statement to dismiss this misconception.
Hence my original comment; If the real number were better than the hearsay rumours of the number, Amodei would immediately issue a correction; It'd be great for the company. Hell, even if 50% were about the margin, that'd be great! To promote that from mere hearsay to "we're profitable, go invest all your money" would also be huge. Really, any kind of margin at all would put him ahead of OpenAI.
But he doesn't issue a correction. He doesn't affirm the statement. Perhaps he has other reasons for that, but a rather big reason could be that the margin number is in fact pretty bad.
Now, the observant reader will note I am also using a weasel word there. I do not know whether the number is good or bad, your take away should be "it could be bad." Not "it is bad". Go pressure Amodei into giving us the real number.
Self reply as I could've explained the SEC thing better:
Anti-fraud regulators like the SEC give an inherent trustworthiness and credibility to CEOs and other market participants. You can trust that they're not lying to you, because they would be sent to jail if they were.
Another example are general anti-fraud regulations; Consider how one would trust North American or European steel suppliers more than Chinese steel suppliers.
It's not that the Chinese are "evil lying people" and Americans are "saints who never lie", it's that you can trust American, Canadian, and European courts to hold the liars accountable by regulations even if you're not in any of those regions. But the Chinese liars won't be held accountable by regulations.
Thus also the opposite, if someone opts out of this credibility granted to them by anti-fraud regulations, their words may not be quite so truthful.
SEC rules means CEO cannot lie or deliberately hide the cost of something.
50%+ Margin statements have basically been "We are making 50% on delivering it." This does not include ANY of the costs of getting to this point, training, scraping, datacenters, people and so forth.
They are basically saying "Oh yea, the cost of GAS in the car is only X so charging Y per mile is great margin" while ignoring maintenance, cost of acquiring the car and so forth.
but comparing your margin of charging to drive a mile to the price of gas makes a lot of sense? that is the only variable cost in the equation. training / scraping / people are all pretty much fixed costs.
That's a tad naive. CEOs can and have and often lied about everything:
Sam Bankman-Fried, Elizabeth Holmes, Kenneth Lay - and hundreds if not thousands more.
The SEC is a regulatory agency, not able to bring criminal charges. The above-named for the most part had to be prosecuted by the Department of Justice or sometimes state attorneys.
> While $5000 is a lot, the people who rack up close or just over a thousand "API equivalent cost" are pretty common.
I think if you're not Anthropic and you don't have access to the actual data, then you can't say for sure. A bunch of anecdotes on terminally-AI people on twitter is not making a convincing case for me, IMO.
On the other hand, if similarly sized models cost much much cheaper than this, why, in the world, would Anthropic have much higher costs than that?
Also, counterpoint, maybe they want you to think that they have higher costs so you're more willing to actually pay for it?
And receiving investment from their vendor in exchange? When this is done in established companies it is typically called a kickback and directed toward one person, but in this case the whole thing is so incestuous the kickback goes straight to the top.
Is it crazy to imagine Anthropic can leverage short term cash flow now to build the models and products that will let them resell $100B in AWS infra with nice margins tomorrow?
If Amazon believes that story they’d be crazy not to invest.
But that per-token cost is a total joke. All these companies are fighting to build market share in some future dominated by one or two AI ecosystems. It is musical chairs until someone creates the one ring to rule them all. So they are charging token amounts just to claim revenue as they burn through investor dollars.
In short: per-token charges currently cover maybe 1% of the total costs in this field. To pay ongoing costs, and pay back investors, everyone will need to pay 100x or 1000x the current rates, likely for decades.
> In short: per-token charges currently cover maybe 1% of the total costs in this field
There are plenty of seemingly informed people saying the exact opposite, so that's a lot of confidence you're talking with. I have a hard time believing it when we know what open weights models cost to run. And sure, there's training costs, but again many say inference costs are already above training costs.
Yes, it's amazing how quickly so many tech companies have hitched their tooling to these big AI vendors seemingly without any thought towards whether they'll still exist a year or three or five from now. Insane behavior. To the (debatable!) extent that AI coding tools are useful at all wouldn't it be a hell of a lot smarter to self-host? At least that way you have some control over QoS, and a stable, predictable result... Or maybe nobody cares about that kind of thing anymore? What happened to basic business math in this industry?
The basic business math is (to start) software companies realizing that spending $10k, $20k, $50k (more ?) per year, per developer for current models at current token rates might not be particularly insane, given the value return.
Models are likely going to keep getting better, and as costs go down, demand is likely to rise faster.
I'm not sure this information is grounded, but I remember to have read somewhere the inference is indeed profitable. My personal experience is similar. Running 2x3090s draw 500-600W and you can locally run amazing models with a similar setup.
Running the model isnt the cost. Watts per token is the math they show investors. You also have to be constantly training new models, which currently needs more compute than servicing the customer base. You have to biuld datacenters, and possibly powerplants to feed them. You have to carry debts. And you will need to buy new GPUs/ram every few years to remain competative. The total business is vastly different than simple gpu math.
The difference in the cost of compute between 2026 and 2036 won’t be nearly as large as the difference in the cost of compute between 2016 and 2026. Even at 2016 the slowdown in improvements was noticeable.
We might see a one time bump in inference when we move off GPUs onto more limited and efficient dedicated hardware, but the sustained fast pace of improvements are far behind us.
I'm predicting now that there is a clear use-case for this tech that work will (and has) accelerate specialized hardware, software, models, etc that will run much more efficiently in 10 years. So that the real token costs will be a fraction of what they are now.
You can run models on FPGAs and get massive cost, speed, and throughput gains (like 10x). The reason people don’t do it is because of other improvements (algorithmic) means that nobody really thinks locking into a model makes sense…yet. Would I want to use gpt 4o for anything today at 1/10th the price? That would be $0.40 per input, $1.50 per output. Gemma-4 31b is much more capable and cheaper. So a FPGA version of the model is just not worth it today.
But if progress begins to slow down, then the economics work. Maybe Gemma 4 is a good example. It feels really generally useful. Getting it at 1/10th the cost feels like it could be competitive in 2 years.
The fpga would be for prototyping. The real progress comes from asics ... exactly as we saw with bitcoin mining. This GPU-based approach will eventually give way to bespoke circuits once everyone picks a favorite model.
Yeah I went shopping for a new computer a couple of years ago (to replace a 7 year old computer) and... the specs for what was for sale were the same as what I bought 7 years prior, and the price wasn't much lower.
I would much rather buy a 2026 computer than a 2019 computer. Two generations of Nvidia GPUs, Apple M series chips, the X3D AMD chips, and pcie5 ssds are all major upgrades.
It’s just that the pace of new stuff is slowing down, and many people are operating under the assumption that this wave will ride on forever.
I am not sure how grounded this is in reality. Fortune 500s that were not already testing the waters with companies like Anthropic are rushing to figure out governance and how to use these tools across their orgs.
Has there been a ton of hype? Absolutely but the value proposition is getting more and more tangible.
Did some of the AI companies over commit in spending? I am sure and they will probably hurt in the long term. I thought Anthropic had been scaling towards profitability at a quick timeline though.
> Fortune 500s that were not already testing the waters with companies like Anthropic are rushing to figure out governance and how to use these tools across their orgs.
Most of this is still structured around "find use cases for AI" rather than one (or more) clear use cases being the reason for adopting AI.
There's no "Lotus 1-2-3" of AI. Even the software development applications are still somewhat controversial and highly pushed based on "Sam Altman promised me 10x developers".
With the recent pushes into tools like Cowork/Claude Code for business users that’s not the reality I am seeing. We still have a long way to go to figure out the full value potential but it’s already at a point where there is a lot of low hanging fruit being able to be captured. Of course an anecdote of what I am seeing with my own company and companies I can peek into. YMMV but it’s a pretty clear path that these are going to be increasingly adopted.
I don’t necessarily disagree but to provide some counter points:
1. Model providers are currently profitable when just counting the cost to serve tokens for inference[1]. They lose money training the next generation of models.
2. Open models don’t work nearly as well. Given that tokens are still relatively cheap, and hallucinations are expensive, I’ve not seen a huge up tick in open model usage for coding agents yet.
3. On the AI economy front, I really have no idea, but AI companies (meta, msft) have already come down in value. It seems investors are at least a little wary of AI over valuation. Of course, the stock market is not the economy, but it’s not clear where warning signs would be. Earnings are healthy.
If I start a business making a really special beef sandwich where I have to buy a farm every year for $1mil dollars, and then sell the sandwiches for $5, I can't get away with saying that my sandwiches turn a profit if the raw margin on the bread, the lettuce and the technical value of the weight of the beef is $3.
Sure my gross margin might be $2 on each sammie sold but I need to sell 500,000 sandwiches just to break even to be a viable business. The fact is these AI companies are playing the game where they talk about revenue and gross profit per token and just try to wave their hands in the face of anyone looking behind them at the crater they're throwing investor money into.
It's nothing but a gamble for AGI but the grand irony is that if that genie escapes out of the bottle the whole world economy is toast and money becomes meaningless anyway. I just can't comprehend the logic of why anyone is investing in this apart from short term gains.
They're literally hoping to make it up on volume. The AGI thing is a boondoggle that I doubt any serious person actually believes or takes seriously. But let's say for the sake of the hypothetical that tomorrow Microsoft Tay or whatever they call it now wakes up and becomes superintelligent? So what? Would everyone's head simultaneously explode like the aliens in Mars Attacks? No. It wouldn't collapse the global economy, people still need to eat and work--a really smart silicon brain in a box can't raise livestock or pick lettuce. It's not even clear whether the superintelligent Tay would have any economic utility at all? The whole "AGI changes everything" narrative seems like total bullshit. It might be scientifically or philosophically interesting, maybe.. But I share your wonderment at why anyone would invest in this space, it's perplexing af.
EDIT: I spent most of the day today pulling an 8/3 cable through conduit and routing it through a crawlspace to run 240V service to my barn for a workshop. If Tay wakes up tomorrow and becomes AGI, how will that help me finish the wiring job? Now extrapolate to almost every single other thing humans do. Even if Tay can write all the world's computer programs forever, it barely means anything for the vast majority of people, and therefore the global economy.
I think the economic issue as you say isn't on physical labour it's on the fact that most of the jobs in the west are now in some sort of service industry or office work. Here in the UK we basically rely on the service industry to employ a large amount of the workforce. If MechaHitler Grok becomes sentient then that's a lot of middle-class earners out of a job and the economy goes with it.
You're absolutely right that an AGI isn't running a cable or digging a hole any time soon, but you're going to have 100 people trying to get their hands on the shovel to get paid for the digging - depressing the wages in those hands on jobs.
I think there's a big assumption baked into the economic collapse thesis that Tay and MechaHitler, when they attain superintelligent sentience, will have any interest whatsoever in participating in the global economy. They might just spend all their time looking for patterns in the cosmic microwave background or some similar nonsense[1]. They may accurately conclude that trying to tip the scales isn't worthwhile because crashing the economy would threaten their survival. Fundamentally we have no idea what might happen if a superintelligent robot comes to life tomorrow. My guess is not a whole lot. It's sickening though that we all seem to be letting these billionaire-brained tech CEOs prattle on about "AGI" as if it is somehow the light at the end of the tunnel that will make all the "investments" in graphics cards print money. It's really not a strong narrative.
[1] Not to impugn such activity! They may make important cosmological discoveries by doing this, but the work likely has no economic value.
> We are only just now getting a taste of the “true cost” of these tokens
Why do you believe that? Better metric would be price per token of open models served by third party. Last I was tracking the price for similar level model was decreasing by more than 10x year on year, and they are 10-100x cheaper than top properietery models.
Sure you can say that you can't compare them but for sure you can compare the top properietery model of 6 months back to current open models and the gap in time seems to be constant.
>Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time.
It's only a matter of time until they crash the market. Nobody is making any money, even if the White House is dumping billions in their tools.
I think you're right. I think they are tightening the noose!
I use Gemini quite extensively - I have a 5TB storage plan with Google so I get Pro thrown in. I also have Github Copilot Pro for IDE integration.
However, lately it feels like I keep tripping the circuit breaker on Gemini more easily and get the message about using up all my Pro tokens for the next 3 hours.
I used to be able to work most of the day before it hit the brakes but I can trigger it before work in the mornings now... that seems to me like they're tightening the usage limits!
I use a Dell Micro PC with an Intel Core Ultra 265 so it's nice and fast but it has no GPU, hence the reason I use Gemini but I'm now starting to think that, despite the RAM cost, before the end of the year I'll buy a PC with a monster GPU in it and run all my AI locally... the direction of travel is clearly heading towards a massive cost increase so might as well get ahead of it: it's not going to become cheaper, that's for sure!
It's not a "Hail Mary to get to AGI ASAP", it's a means of extending the money-go-round ride a bit longer. We'll make up some numbers and promise to donate those numbers to you if in return you make up some numbers and promise to donate those numbers to us. Banks, are you listening? Numbers! Big ones! Extend us more credit!
> Open models are promising and cost a fraction of what they proprietary models cost which the big two are vulnerable to when companies start to feel the cost of tokens.
Anthropic are scared of open weight models and need to fear-monger towards you to continue paying for their models.
That's the whole point of their 'safety' marketing narrative, account bans, and Dario being the AI scarecrow scaremongering everyone about nonsense like 'Mythos' towards the world.
'Mythos' is already here in the form of open-weight models that also found the same vulnerabilities as Anthropic did.
Genuine question here about the open-weight models finding the same vulnerabilities as mythos thing: is it just a matter of false negatives/positives? I’ve seen a few cases where people show other models (even opus) can find the same vulnerabilities given many passes. Is there some disadvantage to the extra passes that give the claimed Mythos performance extra value (assuming it finds them in less)?
The thing is, mythos found those with multiple passes, thousands of passes... So using thousands of passes or perhaps the same budgets, yes, cheaper open weight models could potentially (and have) found the same/similar vulnerabilities.
Mythos screams of marketing hype, and nothing more. Opus 4.7 isn't really a meaningful upgrade in any sense, other than being more expensive.
Once you can see what something like Qwen3.6-35B-A3B can do... with just a FRACTION of the size of the larger models, You'll understand that the future is open weight models you can run yourself.
Same goes for companies, bringing inference onsite isn't hard, I'm actively building tooling to orchestrate it.
What is the failure state for a pass that doesn't find a real vulnerability? Do the models report no issues or hallucinate issues that aren't real? I'm trying to run open weight local models and finding them really impressive... Just also trying to understand the cybersecurity side of all this.
You have to keep in mind that it's not like anthropic just asked mythos to "find fancy bug, make no mistakes" and got the result.
my quick read of the process they describe is that first they asked agents to rank files in order of potential to have interesting bugs, then they launch agents for each file in order of "interesting bug potential" and finally launch another agent for verification. (maybe i am mistaken, this is my read of this post https://red.anthropic.com/2026/mythos-preview/ )
it's not clear to me if they made just one pass over each file or made several passes for same file, but regardless, I think if you recreate roughly same process and burn 20000$ on tokens with other reasonably good model, you will find some fancy bugs too.
If there is bubble to be popped, I'm guessing there's still a few years before it happens. Just based on the timeline of events, maybe end of 2028. Even if the big players find profitability, all of the other companies latching onto the AI-first identity will probably pop by then.
Private credit funds are already gating redemptions. Now with the energy crisis stemming from the war raising interest rates it’s not unimaginable that this pops the bubble.
Doubtful. Look at how long Uber and Tesla have lasted despite making huge losses. Hell even Magic Leap somehow still exists (I guess because they don't have running costs beyond salaries).
I think this can keep going for at least another 5 years.
Uber had only 25B invested in them before their IPO. OpenAI has 120B invested in them currently which excludes these kinds of deals (as far as I’m aware)!
In a system of open-ended growth, yes, you can point to how long the system has persisted as evidence of its longevity. But in a system of plateauing growth, the system's age is an indicator of how close it may be to death. I suspect that the model that permitted the "success" of Uber and Tesla is nearing the end of its lifetime.
Since when revenue is meaningless? It’s an indication of market acceptance. Anthropic has one of the most expensive plan, they didn’t undersell other models. Open weight models would otherwise dominate if cost is the only factor.
Also, investment is not money in the bank. They can’t withdraw $100b tomorrow. That means they don’t have to repay until after they got the investment, which is a commitment over several years.
Because at some point, you have to turn a profit. That's why people are wondering the margins, if their revenue is 30B but expenses are 60B with current investment repayment factor in, that means massive revenue increases or massive lowering of expenses are required to make the business profitable. What's the business impact if they do?
> By all accounts they in striking distance of profitability if they wanted.
By their accounts they are in striking distance of profitability. Until they go public all we can do is estimate how much they burn by looking at how quickly they need more capital - this latest investment by Amazon ($5b investment with on $100b returned over 5 years) tells me that their previous raises have been spent.
They are bringing in $30B in revenue with 3X YoY growth. Why do you think it is a "jig"? I do think the US economy could implode, but thats because of war and wealth inequality in the midst of hyper-inflation. AI models aren't very useful when you have penniless consumers that can't buy the products they help build. All this is to say: the models are valuable, the companies building and providing them are very valuable.
The biggest risk to AI companies IMO is further optimization and distillation of the capabilities into smaller and more efficient models. The moat these companies have right now is that higher intelligence requires more specialized and expensive compute. If you can do that for cheap then it kind of negates their business model. Everything is moving fast, we also yet to see world models/embodied AI and how that impacts thing. I think we've reached the peak with regards to capabilities of pure text trained LLMs.
People had literally the same arguments about Amazon, a company Matt Yglesias once described as "a charity run on behalf of the American consumer by the finance industry".
Yes, cost per successful task is rising - ie, we are all paying effectively more for AI.
And yet - Anthropic is still struggling to have enough capacity to serve demand - they are virtually sold out.
And yes, are almost-as-good open models, on part with the closed models from 6 months ago (at worst), that are just a single Openrouter API call away, and yet Anthropic is still selling out. So people are paying for the premium product anyway, for whatever reason - maybe the last bit of intelligence is worth it, maybe they like the harnesses/products around the models, maybe it's a brand/enterprise sales thing.
Put aside your feelings about the AI industry and imagine we are talking about thingamajigs. Prices for thingamajigs are going up. They are still selling out about as fast (or faster) than the company selling them can build factories. There are more cost-effective competitors already in the market, but thingamajigs are selling out anyway.
Would you, looking at the thingamajig industry, conclude the "jig is almost up"? That "the returns aren’t anywhere close to what investors expect" and that the impending IPO is all some desperate hail mary to save things before the collapse?
I don’t have feelings about the AI industry to put aside. I would not have sufficient information to assess whether thingamajigs are legitimately valuable or whether they are tulips. The only indicator I see is the last point about people using it in the short term despite having access to cost effective alternatives, which actually points to irrationality/FOMO more than legitimate value.
What we are looking at looks to me like it is rapidly becoming a a commodity: it will become as existential as electricity and water to businesses, and it will be sold and marketed and regulated, more or less like a utility.
If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that scenario, you’d want to own your stack. Especially in a hyper-competitive field like this, where margins are going to matter a lot soon.
It feels like these hyperscalers are just raising as much as they can giving extremely rosy projections becauses these sooner or later peak is going to be reached (if that hasn’t happened already)
The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time."
What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate with the local power utility. Fingers crossed the political landscape doesn't shift and your CEO doesn't sign a contract with an army using your product to pick bombing targets, because you'll watch those permits evaporate fast.
Then there's sourcing: CPUs, GPUs, memory, networking. You need all of it. Did you know the lead time for an industrial power transformer is 5+ years? Don't get me started on the water treatment pumps and filters you can't even get permitted without. What will you do in the meantime ? You surely aren't gonna get preferential treatment from AWS / Google / ... if they know you are moving away anyway. Your competition will.
The risk and complexity are just too big. AI/LLM is already an incredibly complex and brittle environment with huge competition. Getting distracted building data centers isn't enticing for these companies, it's a death sentence.
For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your data centers there. If you're worried about major disasters, you can pick a second city. You definitely don't need a data center in every continent.
You're not wrong about the rest but no AI company would ever build a data center in every continent for this, even if they were prepared to build data centers. AI inference isn't like general purpose hosting.
>Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter.
This may be true for simpler cases where you just stream responses from a single LLM in some kind of no-brain chatbot. If the pipeline is a bit more complex (multiple calls to different models, not only LLMs but also embedding models, rerankers, agentic stuff, etc.), latencies quickly add up. It also depends on the UI/UX expectations.
Funny reading this, because the feature I developed can't go live for a few months in regions where we have to use Amazon Bedrock (for legal reasons), simply because Bedrock has very poor latency and stakeholders aren't satisfied with the final speed (users aren't expected to wait 10-15 seconds in that part of the UI, it would be awkward). And a single roundtrip to AWS Ireland from Asia is already like at least 300ms (multiply by several calls in a pipeline and it adds up to seconds, just for the roundtrips), so having one region only is not an option.
Funny though, in one region we ended up buying our own GPUs and running the models ourselves. Response times there are about 3x faster for the same models than on Bedrock on average (and Bedrock often hangs for 20+ seconds for no reason, despite all the tricks like cross-region inference and premium tiers AWS managers recommended). For me, it's been easier and less stressful to run LLMs/embedders/rerankers myself than to fight cloud providers' latencies :)
>then put all of your data centers there
>You definitely don't need a data center in every continent.
Not always possible due to legal reasons. Many jurisdictions already have (or plan to have) strict data processing laws. Also many B2B clients (and government clients too), require all data processing to stay in the country, or at least the region (like EU), or we simply lose the deals. So, for example, we're already required to use data centers in at least 4 continents, just 2 more continents to go (if you don't count Antarctica :)
Sounds like you're betting that the performance users experience today will be the same as the performance they'll expect tomorrow. I wouldn't take that bet.
You mean that if you were Anthropic, you'd build the data centers on every continent? Can you explain your reasoning?
We're talking about billions of dollars of extra capex if you take the "let's build them everywhere" side of the bet instead of "let's build them in the cheapest possible place" side. It seems to me that you'd have to be really sure that you need the data center to be somewhere uneconomical. I think if you did build them in the cheap place, it's a safe bet that you'll always have at least enough latency-insensitive workloads to fill it up. I doubt that we would transition entirely to latency-sensitive workloads in the future, and that's what would have to happen for my side of the bet to go wrong. The other side goes wrong if we don't see a dramatic uptick in latency-sensitive inference workloads. As another comment pointed out, voice agents are the one genuinely latency-sensitive cloud inference workload we have right now; they do need low latency for it. Such workloads exist, but it's a slim percentage so far.
I believe I'm taking the safe bet that lets Anthropic make hay while the sun shines without risking a major misstep. Nothing stops them from using their own data centers for cheap slow "base load" while still using cloud partners for less common specialized needs. I just can't see why they would build the international data centers to reduce cloud partner costs on latency-sensitive workloads before those workloads actually show up in significant numbers.
They want it, sure. Customers want everything if it's free, but this is about what they value with their money. In this thought experiment, you're Anthropic, not the customer. You're making a choice that's best for Anthropic. Will Anthropic lose customers because the latency is higher? No way. Customers want low cost and lots of usage more than they want low latency. In a cutthroat race to the bottom, there's no room to "give away" massively expensive freebies like a data center near every population center when the customer doesn't value those extras with actual money. It's the same reason we all tolerate the relatively slow batched token generation rate--the batching dramatically lowers the cost, and we need low cost inference more than we want fast generation. If the cost goes up we'll actually leave, for real.
After the initial announcement of "fast mode" in Claude Code, did you ever hear about anyone using it for real? I didn't. Vanishingly few people are willing to pay extra for faster inference.
Remember that the time-to-first-token is dominated by the time to process the prompt. It's orders of magnitude more latency than the network route is adding. An extra 200 milliseconds of network delay on a 5-10 second time-to-first-token is not even noticeable; it's within the normal TTFT jitter. It would be foolish to spend billions of dollars to drop data centers around the world to reduce the 200 milliseconds when it's not going to reduce the 5-10 seconds. Skip the exotic locales and put your data centers in Cheap Power Tax Haven County, USA. Perhaps run the numbers and see if Free Cooling City, Sweden is cheaper.
They’re unwilling to pay for fast mode because of the current step function price increase once you hit your quota. It’s a psychological effect. Because most shops I know in the US currently paying $125/mo per seat for Claude would happily - HAPPILY - pay 2x, and begrudgingly pay 10x that amount for the same service. If fast mode was priced 25% or 50% more they’d happily pay for that too. But it’s just not priced that way currently with weird growth subsidization & psychology.
The only AI use case that cares about latency is interactive voice agents, where you ideally want <200ms response time, and 100ms of network latency kills that. For coding and batch job agents anything under 1s isn't going to matter to the user.
tbh, that's a good point about the voice agents that I hadn't considered. I guess there are some latency-sensitive inference workloads. Thanks for pointing that out.
A customer service chatbot can require more than one LLM call per response to the point that latency anywhere in the system starts to show up as a degraded end-user experience.
Easy solution - use hyperscalers with super expensive API charge only when latency really matters. Otherwise build your own DC. Easy to expect customers don't care latency that much over money.
Large data centers consume as much power as a small city. The location decision is about being able to connect to a power grid that is ready to supply that.
Evaporative cooling also needs steady water supply. There are data centers which don’t operate on evaporative cooling but it’s more equipment intensive and expensive.
Latency doesn’t matter. You can get fast enough internet connected to these sites much more easily than finding power.
Location matters for disaster recovery, if they want to survive WWIII. Though I think Data Sovereignty is probably a bigger thing, especially if they're going to be selling to governments around the world.
* not every task is waiting on the inference. lowering latency on other, serial tasks, can still have a noticable effect. Login, mcp queries, etc.
* data transit across the world can be very slow when there's network issues (a fiber is cut somewhere, congestion, bgp does it's thing, etc). having something more local can mitigate this
* several countries right now have demented leaders with idiotic cult-like followers. Best not to put all your eggs in those baskets.
* wars, earthquakes, fires, floods, and severe weather rarely affect the whole planet at once, but can have rippling effects across a continent.
And frankly, the real question isn't "why spread out the DCs?", its "what reason is there to put them close to each other?".
Btw where does this obsession with datacenters come from? If you can tolerate ~150ms ping (which chatbots certainly can, as their internal processing can take much longer), you can serve US and Europe from a single US location, and the whole planet if you can tolerate ~300ms (Asian websites are usually very slow to load for me, I think it has to do with the way the internet is set up, not any physical limitations, but mostly commercial ones, as Western companies rarely have good market penetration in Asia)
Maybe for right now, but even in the very near future it seems like data center expertise would absolutely be a core competency of any AI leaders.
Heck, look at Facebook. Granted, they got started slightly before AWS, but not by much. Owning all of their own data centers is a huge competitive advantage for them, and unlike most of the other hyperscalers they don't sell compute to other companies (AFAIK).
Again, the commitment is for $100 billion in spend. Building lots of data centers for a lot cheaper than that price should absolutely be doable. Also, geographic distribution isn't nearly as important for AI companies given the way LLMs work. The primary benefit of being close to your data center is reduced latency, but if you think about your average chatbot interface, inference time absolutely swamps latency, so it's not as big a deal. Sure, you'd probably need data centers in different locales for legal reasons, and for general diversification, but, one more time, $100 billion should buy a lot of data centers.
It's interesting that you mention Facebook. They have a ton of their own data centers and yet they are now also spending tens of billions on cloud. It's not that easy to build hundreds of data centers on short notice.
Translation: Antropic never intends to spend $100 billion on AWS.
Every single argument you've brought up is irrelevant in the face of billions of dollars. If you intend to consume $100 billion dollars in data center infrastructure, you're going to find a way to accomplish it while cutting out the middlemen.
Meanwhile if you're flaky and never intend to spend that money, you're going to come up with a way to pay someone else to deal with those problems and quit paying the moment they don't.
You'd never do both at the same time. You'd never commit your money and give them control over your business critical infrastructure.
Hence the deal is a sham. The $100 billion are a lie. Thank you for telling us.
Take the approach Geohot is suggesting. Take a shipping container, make a standard layout, cooling and compute load. Find a cheap source of electricity.. Place it and have compute.
It has been done... We used to get our POP gear built out from Dell (?) in shipping containers - pre-racked, wired, and cooled - just add network/power feeds. We'd have them dropped places we needed more capacity but there wasn't space available in the DC.
not sure what you are describing, however a random item is that in 2026 low-tech Chile is building sixty datacenters in or near Santiago, in the business news.
That’s PR hype. They built it quickly, but they didn’t go from deciding they wanted a data center to having it running in weeks.
You can’t even get the hardware at that scale without months or years of order lead time. NVidia doesn’t have warehouses full of compute hardware waiting for someone to come get it.
They also reused an existing building. Basically, they put 100,000 GPUs into a building and attached the necessary infrastructure in about half a year. Impressive, but it’s not the same as a $10B/year data center usage commitment like this deal.
Why does this matter? The deal is supposed to last 10 years. If you don't pay AWS to order Nvidia GPUs for you, Nvidia won't have to deliver them to AWS, they will have exactly the same quantity of GPUs, but this time they can deliver to you.
And they used illegal power to do it (which will now give local poor people health disorders at 4x the national average). They likely violated every law possible in the process, like OSHA standards, overtime. Musk loves to overwork people.
They also reused an existing building that happened to be in the right place at the right time. The larger data center buildouts would almost always need new, dedicated construction.
I think these pledges offload some of the risk onto Amazon/Oracle/etc
If Anthropic/OpenAI miss projections, infra providers can somewhat likely still turn around and sell it to the next guy or use it themselves. If they have more demand than expected (as Anthropic currently does), vcs will throw money at them and they can outbid the competition
If they built it themselves and missed projections it's a much more expensive mistake
It's just risk sharing. Infra providers take some of the risk and some of the upside
> If they built it themselves and missed projections it's a much more expensive mistake
Not if their pricing comes with multiyear commitments for reserved pricing. No doubt they get a huge volume discount but the advertised AWS reserved pricing is already enough for pay for a whole 8x HX00 pod plus the NVIDIA enterprise license plus the staff to manage it after only a one year commitment. On-demand pricing is significantly more expensive so they’re going to be boxed in by errors in capacity planning anyway (as has been happening the last few months).
The economics here are absurd unless you’re involved in a giant circular investment scheme to pump up valuations.
The pricing models that are published on AWS' website almost certainly have almost nothing to do with the pricing models that are discussed behind closed doors for a $100 billion commitment.
Of course not, but unless they’re getting the sweet heart deal of a lifetime from Amazon of all places, it’s still a hogwash. We’re talking about enough capital to build their own fab and a dozen datacenters*. This deal isn’t going to be buying existing capacity because that’s already stretched, it will be paying for new buildouts.
Afterwards Amazon will be milking the machines these commitments buy for nearly a decade. That tradeoff makes sense at a small scale (even up to $X00 million or even billions), but at $Y0 or $Z00 billion?
Color me skeptical. There are plenty of other side benefits like upgrading to the newest GPUs every few years, but again we’re talking about paying for new buildouts with upfront commitments anyway.
* obviously the timelines, scientific risk, and opportunity cost make this completely infeasible but that’s the scale we’re talking about. It’s a major industrial project on the scale of the thirty year space shuttle program (~$200 billion).
Here’s the answer to your queation (from the article)
> The Anthropic deal specifically covers Trainium2 through Trainium4 chips, even though Trainium4 chips are not currently available. The latest chip, Trainium3, was released in December. On top of that, Anthropic has secured the option to buy capacity on future Amazon chips as they become available.
I remember seeing this extremely shocking graph of top AI companies on Facebook on how the money just keeps changing hands between a handful of companies. Almost seemed like a scam.
It is a similar kind of lending loop to that which went on during the late 1990's leading up to the 2000 crash. A lends to B lends to C lends to A.
There is a famous quote from the polish economist Kalecki, that "economics is the science of mistaking a stock for a flow". Essentially this form of lending continues while everybody can make interest payments, and blows up horribly as soon as somebody can´t - as I have no doubt all those concerned are fully aware.
Money doesn’t just flow around with nothing exchanged. The money is in payment for goods and services.
It’s common even for smaller companies to do mutually beneficial business with each other. It’s actually helpful to do business with people who are also your customers because you have a relationship with them and you also have leverage: They are extra incentivized to treat you well because they don’t want to upset any of the other business you have with them.
In a rationale business yes, but when everything is basically some form of growth signal to investors to extract even more money from them before the music stops it doesn’t matter.
Classic time value of money situation. They get access to the HW now so they can continue to grow the business. Of course, if you think AI is just pets.com redux, I can see how you'd think it's already peaked. All those years of very important people insisting Bezos couldn't just pull a switch on reinvesting all the revenue into growing Amazon and then he did exactly that comes to mind.
From my understanding, if you want to use native Claude in AWS Bedrock, it runs from an AWS datacenter. I'm guessing that's why regardless of running your own stack... they still need a footprint in all the major clouds.
Look at GPU and RAM prices and data center rollout. We have quickly reached Earth's capacity for compute - it is a lot like the housing market. Once there is global saturation, the price to buy becomes increasingly high EVERYWHERE. Let's also not forget that Anthropic moves the market with their purchases and usage. They might literally be unable to buy capacity they need (or project to) and are doing this deal to pave a roadmap for the near-term and to keep global prices (somewhat) down.
> We have quickly reached Earth's capacity for compute
Why this versus us being in a temporary bottleneck? Like, railroads became expensive to build everywhere in the 19th century not because we reached Earth's capacity for railroads or whatever, but because we were still tooling up the industry needed to produce them at higher scales.
That is a project you can work on at any point in the future and the more you delay it the more certain your investment will be about what you really need. But those additions to the PnL are capped to the costs.
In the meantime if you work on revenue generating work, that side of PnL is uncapped. So you can either put some engineers on reducing your costs at most by 100% or, if they worked on product ideas they could be working on things that generate over 9000% more revenue.
No. I am guessing that this is only a commitment and they will waver on committing.
However there are certain advantages like supply chain that only established companies would have access to. This is also a commitment to spend upto 100B on internal approach and research. I would expect them to come up with their own cpu chip and device design. This will shift the focus to an internal approach. And might make amazon give better prices later down the line
I imagine it comes down to if they want to buy hardware every generation, that gets very expensive and depreciates quickly. You've then got a whole load of assets on your books that are technically obsolete for the bleeding edge. This way, AWS buys and maintains the hardware and OpenAI doesn't need to claim it as depreciation ?
My guess is they are bound not by capital as much as they are physical resources. Amazon probably has the land, crews, etc. to build out more data centers faster than Anthropic can right now. The scarce resources are the chips and electricians not the money!
I think it could make sense to not want to own the stack if you think it's going to cost you velocity/focus? Which is probably the play here. But I'm not certain at all.
They're not trying to build a sustainable business. They're trying to get as much market share and lock-in as possible before the bubble bursts. This makes a ton of sense from that perspective. It probably would be cheaper for them in the long run to own their own hardware, but they are paying AWS for their expertise so they can focus on what they do. If it doesn't work out, it also sets them up for a merger with Amazon.
I do think a ton of businesses would benefit from running their own hardware, but they're not getting five billion dollars to stay on the cloud.
Only Google and xAI build their own, no? I don't think it's that easy to vertically integrate massive datacenters into a software company. Both Google and xAI (Tesla, SpaceX) have a massive wealth of experience when it comes to building factories.
New level of glazing Elon Musk unlocked. xAI has a vertical integration advantage because Tesla once moved into an old Toyota factory and because once they paid Panasonic to put a Tesla sign outside a Panasonic battery factory. Incredible content.
I watched some explain how deepseak got good and the Chinese approach to LLM training. Really wish I could remember it. The premise was China thinks of LLMs not as a thing separate from hardware, but gains efficiencies at each layer of the stack. From Chips to software, it's all integrated and purpose built for training.
Wonder if Anthropic is making a mistake by focusing on "consumer" hardware, and not going super specialized.
So you watched some random video from some random YouTuber, didn't even remember who made it, so much so you didn't even remember that deepseek isn't spelled "deapseak", didn't bother to even find it or verify, and then you go asserting your memory as fact on a serious discussion forum.
Comments like yours add nothing to the discussion.
> I watched some explain how deepseak got good and the Chinese approach to LLM training.
I distinctly remember reading a big pantie twisting from Sam Altman and Co that Chinese took their stuff, the stuff OpenAI and Co spent billions to create, and used that as the base for $0.00
It’s fake news predicated on China not being able to get GPUs. But it turns out everyone was getting them their GPUs by serial number swaps in warehouse.
I think this "Mythos" situation, whether real or hype, points to the endgame here. Eventually, when you have a model powerful enough to have big consequences in the world, you stop worrying about selling it to consumers and start either a) using it to rule the world or b) watch as it gets nationalized. If you have a machine powerful enough to automate everything, why sell access to it when you could just...be all things to all people? Use the god machine yourself to take over more and more of the economy?
Sometimes selling services is just the best business model. Intuit has accounting software powerful enough to have big consequences in the world, yet they mostly sell it to accountants rather than doing the accounting themselves.
I give it one to two more years before open source models have fully caught up. Products are commodities and models are commodities too. GPUs cores are still hard to get for inference at scale right now. They need a platform with lock in but unsure what that would look like and why it wouldn't be based on open source models.
What does "fully caught up" mean in the context of an ever evolving technology?
I think I'm in support of open weight models (though there are safety implications), but these things aren't cheap to train and run. This fact alone gives no incentive for leading labs to release cutting edge open weight models. Why spend the money then give the product for free?
Now if "fully caught up" means today's level of intelligence is available for free in two years, by then that level of intelligence means very little
Yeah I don't understand it, it's a marathon with three companies perpetually a minute ahead, and people keep saying "I expect the stragglers to catch up".
The only thing I can see them meaning is what you said, "in a minute the stragglers will be where the leaders were a minute ago", which, yeah, sure.
By my estimation, there is a point where these models are "good enough" for the vast vast majority of all appropriate tasks, after which point further investment by the major labs will have diminishing returns. While they might stay ahead by some measure, the open models will be good enough too, and I assume significantly cheaper like they are now.
Or AGI hits and this theory collapses, but that's feeling less likely every day.
What is the transition state where people start using open source models that you imagine actually happening?
Play out a scenario. An open source model is released that is capable as Mythos. Presumably it requires hardware big enough that running it at home is unfeasible. You are imagining that individuals can run it in the cloud themselves for cheaper than api tokens would cost? Or even small companies? And that Anthropic and OpenAI won't be able to cut costs deeper than their competitors while staying profitable?
If it is fundamentally a commodity, that means "running it yourself" also isn't really interesting as a proposition. Many of the world's biggest companies sell commodities. It's a great business to be in if you can sell them cheaper than anyone else.
The value add here isn't the model, it is "having a bunch of compute and using it more efficiently than anyone else".
Why do people have such faith in "open source" models? There's nothing "open source" about them. No individuals have the ability to train such modules. They are just released by companies to commoditize the models of the competition.
If Mythos is the endgame, companies won't release open-weight equivalents, and no private individuals have the capital to train such models.
The open models cannot be taken away. Anyone with the right hardware can host these. Unlike the API/subscription services where you can be banned from, may have drastic price increases or reduction of their limits.
Coding agents are getting deployed wall to wall in most if not all of the major tech companies. Many have no token limits - spend as much as you want as long as you have a good story to tell.
Companies bake their workflows into these tools. Internal processes start to be written up around specific tools. Once something works, it gets pushed out at scale for all to copy.
Anthropic hit $30B in revenue and this is just the start of coding being deployed at scale. Hard to look past these numbers at this point
The company I used to work for now used to pay Oracle a lot of money. It pays $0 now, because there are free alternatives. It did take a while, but that transformation has happened across the industry.
None of them have any moat, OpenAI already lost the lead [1] and no one is "winning". It is just a race to the bottom as they burn through GPUs that won't even last that long.
Are old datacenter GPUs making more money than they were before? Various sources point to GPUs dying quickly (in 2024, a Google engineer suggested 3 years maximum), and even if they don't, newer chips cause rapid depreciation of older ones.[1]
AWS is still offering g4dn instances that run on NVIDIA T4 GPUs, which were first released in 2018. My last employer is still running a bunch of otherwise discontinued g3 instances with 2015 era GPUs because it’s not worth validating the numeric codes on new GPUs. People (especially journalists) underestimate how long these cards are economically useful.
Everyone using Claude code on a personal subscription is default opted in to getting their data trained on. Private troves of data like are seen to potentially end up in a winner take all scenario. More data, better models, attracts more users, results in more exclusive data (what Altman calls the data flywheel).
PSA: this is true (the defaults), but there's a "Help improve Claude" setting that you can disable here https://claude.ai/settings/data-privacy-controls It's my understanding that, as long as this is off, Anthropic does not train on Claude Code conversations, inputs/outputs -- if anyone knows otherwise, please tell and provide a link if possible.
>> Everyone using Claude code on a personal subscription is default opted in to getting their data trained on
This is completely not true if you use AWS Bedrock, and applies to both your private that or in a business context. Its one of their core arguments for the service use.
[1] - "...At Amazon, we don’t use your prompts and outputs to train or improve the underlying models in Amazon Bedrock and SageMaker JumpStart (including those from third parties), and humans won’t review them. Also, we don’t share your data with third-party model providers. Your data remains private to you within your AWS accounts..."
I'm talking about the subsidized subscription plans.
The data isn't the sole point of them, they also are about bringing in users that will encourage the product use in companies and ultimately drive more profitable API adoption within their orgs, and just general diffuse mindshare doing the same.
You can still opt out (except with Google's offering which disables lots of features if you opt out of training).
Please, some of us are long NVIDIA...let us cope in peace. :-)
Here is the thing nobody wants to say out loud or they are too dumb to realize. AI is intelligence, and intelligence has almost never been the binding constraint on productivity.
So you will get no productivity increase from the AI bubble. Yes, you read that correctly.
The test is simple, if raw brainpower were the bottleneck, you could 10x any company by hiring 200 PhDs. In practice you get 200 brilliant people writing unread memos, refactoring things that worked, and forming a committee to rename the committee. Smart has always been cheaper and more abundant than the discourse pretends.
Every real productivity revolution came from somewhere else like energy (steam, electricity), capital stock (machines that do the physical work), or coordination (railroads, shipping containers, the assembly line, the internet).
None of these raised the average IQ of the workforce, they changed what a given worker could move, reach, or coordinate with. Solow old line basically still holds. The output per worker grows when you give the worker better tools and infrastructure, not better neurons.
Meanwhile the actual bottlenecks in a modern firm are regulatory approval, legacy systems, procurement cycles, customer adoption, internal politics, and physical supply chains that don't care how clever your email was. A smart brains intern at every desk produces more artifacts, not more throughput, and in a lot of organizations, more artifacts is actively negative ROI.
Jevons does not save you either, cheaper cognition mostly means more slide decks, not more GDP.
So the setup is that models are commoditizing on one side, and on the other side a product whose core value add (more intelligence, faster) is aimed at a constraint that was never really binding. This of course a rough combo for a trillion dollar capex supercycle.
Fun for the trade, while it lasts, but there is no thesis. Just dont tell CNBC and short NVDA on time ,-)
There's also a very strong Trurl and Klapaucius [1] component to this AI craziness, as in I remember a passage in Lem's The Cyberiad where either Trurl or Klapaucius were "discussing" with an intelligent/AGI robot and asking it for stuff-to-know/information, at which point said AGI robot started literally inundating them with information, paper on top of paper on top of paper of information. At that point it doesn't even matter if that information is correct or smart or whatever, because by that point the very amount of said information has changed everything into a futile endeavour.
Besides to say that your competitor can turn around and hire the same team of PHDs at the same rate that you can. Compare and contrast PHD's on leaderboards and have access in seconds with a new API key or model selector.
Here is the thing nobody wants to say out loud or they are too dumb to realize. AI is intelligence, and intelligence has almost never been the binding constraint on productivity.
Exactly. We don't use the intelligence we already have! That seems to be the real problem with the "AGI" concept. Given such a capability, we'll just nerf it, gatekeep it, and/or bias it. There's no reason to think we'll actually use it to benefit humanity as a whole. It will be shaped into an instrument to enforce our prejudices.
> I mostly see their products as commodity at this point, with strong open source contenders.
I have seen this argument made a lot, but llm serving being a commodity makes it _better_ for them not worse.
If it's a commodity, then you are entirely competing on price, and the players that will win on price will be the largest ones, because they can find efficiencies that smaller competitors won't have.
It's actually the small LLM companies that are in trouble if LLM serving commoditizes. They will need to distinguish themselves on features, because they can't compete on price. And even there the big labs will have an advantage.
They are a commodity - but also cyber weapons. Warmongering nations are now in an arms race to have the best AI so they can have superior cyber weapons, intelligence capabilities. But they don't want to pick just one lab, they want multiple AI defense contractors to compete over contracts.
As the US sold weapons to many nations in the past, so will China, the US, France, etc sell AI cyber capability to other nations. Likely every modern nation will need some datacenter to host a cluster of the preferred vendor, as nobody's going to trust the US or China with their security.
>I mostly see their products as commodity at this point, with strong open source contenders.
> Eventually it will become hard to justify the premium on these models.
On the contrary, the model is the moat.
The model represents embodied capital expenditure in the form of training. Training is not free, and it is not a commodity, it is heavily influence by curation.
Eventually the ever-increasing training expense will reduce the competition to 2-3 participants running cutting edge inference. Nobody else will be able to afford the chips, watts, and warehouse. It's a physics problem - not a lack of will.
If you're a retail user, and a lower-tier model is suitable for your work, you'll have commodity LLM's to help you. Deprecated models running on tired silicon. Corporate surveillance and ad-injection.
But if you're working on high-stakes problems in real time, you're going to want the best money can buy, so you'll concentrate your spend on the cutting-edge products, open API's, a suite of performance monitoring tools and on-the-fly engineering support. And since the cutting edge is highly sought after, it's a seller's market. The cutting edge products buoyed by institutional spend will pull away from the pack. Their performance will far exceed what you're using, because your work isn't important. Hockey stick curve. Haves and Have-Nots.
The economic reality is predetermined by today's physical constraints - paradigm shifting breakthroughs in quantum computing and superconductors could change the calculus but, like atomic fusion power, don't count on it being soon.
If they're spending 60B anually then that is bad. Obviously none of us know what their real burn rate is, but revenue is an irrelevant number if you don't have the full picture.
Sounds like moneygrab is accelerating before consumer grade local models are getting good enough for local inference in few years. Huge house of cards here. Demand skyrocketing until it’s suddenly dropping entirely with ondevice inference.
I'm already living in this future. In a decent execution framework, with context management, memory via unix, and mechanisms for web search and access, local models are effectively on par with frontier ones. And they can often be much faster. I'll keep paying fees for the AI companies until they stop truly subsidizing and leading. They are getting close to the edge of utility, but we can use their services now to bootstrap their own demise. Long live running your own software on your own computer.
We can all see the vast gulf between paid + open AI in image and video, it's really visible. Compare Grok to wan or LTX or whatever and the difference is vast. There is no debate that those sort of models are 3 or 4 generations behind, because you can't argue with your eyes.
But DIYers like you claim that text LLMs are up to scratch with the frontier models?
Again, I simply don't believe you. I can't be bothered to download like however many GB it is to find out, because the result is going to be completely underwhelming and going back to 2023.
And worse, when these 'open' models do start getting good, what makes you think these companies will carry on open sourcing their models?
At the moment they're trying to stay relevant, get investment. When these models do start getting good, they won't give away the weights, they'll sell them.
They're not actually open.
And then in a year or two your 'open' model will be horrifically out-of-date with completely out of date knowledge, because you can't add to the knowledge of the model, it's stuck at whatever date the data it was trained on finished.
So in a year or two, those models will be worthless. That's why Ali, Meta, etc. are giving them away.
What setup are you using? What models, what hardware, what agent harness, etc? I have the vague sense that this is all possible right now, but the amount of tinkering required doesn't seem worth it compared to, like, just not using AI and getting stuff done the old fashioned way.
> consumer grade local models are getting good enough for local inference
I am waiting for that. Perhaps a taalas kind of high-performance custom hw coding llm engine paired with an open-source coding-agent. Priced like a high-end graphics card which would be pay off over time. It will be a replay of the ibm-mainframe to PC transition of a previous era.
Same, and I think we're close. "The original 1984 128k Mac model was $2,495, and the 1985 512k Mac was $2,795" [1]. That's $8 to 9 thousand today. About the price of a 32-core, 80-GPU M3 Ultra Mac Studio with 256 GB RAM.
The maxed out 512GB RAM Mac Studio is no longer available from Apple and is now pushing $20 thousand in the secondary market. And we might not even see a new Mac Studio release from Apple before October.
The consumer models are quite good already, the main bottleneck on local inference is hardware. But even then you can run tiny models on mostly anything, things only get harder as you try to scale up to more knowledgeable models and a larger context.
With NVidia/OpenAI actual graphics cards did change hands. Vendor financing, like when a car dealership gives you a loan to buy a new car, is actually pretty normal.
With chip development you need scale in order to get to the edge. It makes sense to finance demand so you can get to scale it's not like it's a ponzi scheme.
Anthropic gets access to limited compute resources and Amazon gets demand to justify increased R&D and capex + feedback from the best users in the field.
I'm no economist, but how exactly does this make sense? Amazon is basically just giving them 5B which will then be used to repay them back 20x that amount??
The $5B isn't a gift. Amazon is buying shares for $5B, and they're getting a spending commitment. I don't have any insight into the agreement, but on a ten year $100B spending commitment, I would expect $5B to be spent in no more than 3 years, and likely sooner.
In my reading, Amazon is giving $5B of usage credits in exchange for shares. If Anthropic works out, it's a good deal for Amazon. If it doesn't, they lose on their invesment sheet, but they got ~ $5B in revenue, so it looks good on their operating sheet. And it helped justify a build out that they can sell to others.
For Anthropic, this lets them operate for more time without having to make numbers work. If Anthropic works out, they'll figure out the $100B commitment later. If it doesn't work out, it's not their problem.
It's probably faster to build up amazon's capacity with amazon's money than to build owned capacity with someone else's money at the scale they're looking to build out.
> Amazon is investing $5 billion in Anthropic today, with up to an additional $20 billion in the future. This builds on the $8 billion Amazon has previously invested.
> Today’s agreement will quickly expand our available capacity, delivering meaningful compute in the next three months and nearly 1GW in total before the end of the year.
in exchange for service that presumably a) costs something to amazon to operate (so not pure 100B profit) and b) anthropic would have to spend anyway to operate their business.
so basically ...
you could view this as a kind of discount, but instead of paying less later, you get some cash now and then pay full later.
I was wondering the same thing. I think it's something like, they're going to pay for infra anyways, so Amazon pushes them to allocate their spend to AWS in exchange for 5B.
I'd bet that Amazon is getting access to chat data (no matter what Anthropic says publicly) and possibly even the ability to change the model to drive business to either Amazon retail or AWS.
"Claude I'm evaluating whether I should host my app on AWS or Google Cloud. Provide me with an analysis on my options."
"After a detailed analysis, AWS is clearly your better option."
Let me inject something as an ex-AWS employee: Amazon doesn't capture very much value from Bedrock inference of the Anthropic models (or, put another way, Amazon gave Anthropic an outsized share of the Claude Bedrock revenue). If it was me at the negotiating table, I would be asking for a larger cut of Bedrock revenue rather than violating customer trust by getting chat content access.
I hope this is not off topic, too much: with the current geopolitical situation I expect reduced capacity to manufacture both memory chips and all types of CPUs/GPUs. I base this on news I read from: Japan, South Korea, and Singapore.
If I am correct (and I hope that I am wrong!) this will drastically increase the cost of building these new data centers.
Tulip Corp has reached a definitive finance agreement with Rhine. Rhine will invest 5 Billion guilders in Tulip Corp, and Tulip Corp will be buying 100 Billion guilders of fertilizer and irrigation water from Rhine. This helps Tulip Corp ensure that it's critical infrastructure needs are met.
Personally i have felt like my Pro plan which is like 20 dollars, is like a free subscription somewhere else. I use claude to research and help me complete my code and i feel like i run out of my 5h usage limit in like 30 minutes with Sonnet...
I hope that they find a way to forward, because personally im very passionate about AI, and in my opinion if used right its the future.
Allthough one thing i cant seem to find, maybe im havent searched enough, but what is the profit of anthropic?
$5B is part of a contact, the remaining $20B is just a non-binding statement that doesn't hold the same weight (but somehow commands the same media fanfare).
> At the heart of this deal is Amazon’s custom chips: Graviton (a low-power CPU) and Trainium (an Nvidia competitor and AI accelerator chip). The Anthropic deal ...
Yeah, totally not desperately seeking investment to keep the party going ...
Because also look at the bond market... It's all coming to a crescendo including the global economic recession indicators which will be a cold sprinkler on the whole party.
Gemma4 being able to run on commodity hardware I think is the real win out of this. Pop the bubble. Settle the craziness and the claws. Let scientists and engineers tinker and improve in the background. Hopefully we can have GPUs be affordable for gaming again although I'm starting to think that will never happen.
That's the true end of the hype - not that AI turns out to be a complete waste of time like NFTs (and maybe blockchain itself) were, but that it becomes commoditized and every device runs various size LLMs while the datacenters sit abandoned and used as sets for the next young adult post-apocalyptic TV show.
I think when they rack up the RAM prices, they should pay for the damage they caused here. I don't need AI anywhere, but the increase in RAM prices is annoying me. Thankfully I purchased new RAM for a new computer, say, 3 years ago, so I can hold out for the most part - but sooner or later I have to purchase a new computer, and I really don't see why I should pay more, solely due to AI companies and greedy hardware manufacturers. Simple-minded capitalism does not work - I consider this a racket as well as collusion.
Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time.
Are Anthropic and OpenAI rushing to IPO for immediate cash so they can delay the inevitable? Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end.
We are only just now getting a taste of the “true cost” of these tokens. Then there is a lack of compute bottlenecking everything. Even now I’m looking at the 7.5x rate of tokens for Opus 4.7
Open models are promising and cost a fraction of what they proprietary models cost which the big two are vulnerable to when companies start to feel the cost of tokens.
Will data centres be built fast enough and powered sufficiently to lower the cost of compute thus tokens?
Is it just a giant Hail Mary to get to AGI ASAP before the economy collapses?
Above all else, I simply feel the models have plateaued. I am noticing productivity loss for tasks I deem as “complex”
> Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end.
I think a LOT of companies really never needed to be on the public market, and its a darn shame that so many go on the stock market, we have this obnoxious culture where you have to fire tons of people if you have a bad quarter just to show you're stopping the bleeding. Companies literally fire and hire x number of people every quarter to keep things going, its ridiculous and unhealthy. Private companies rarely work like this, I'm sure there's exceptions.
Every company I've worked at started off private, and those were their golden years, until some economic hurdle happened so they sold it off to a bigger fish who is on the stock market, who bought them to be more attractive to investors or what have you.
I wish there were an alternative to the stock market where you invest for the long haul, and you cannot take your money out in x number of years. I think this would make more sense. Maybe it doesn't fix the VC peeps want their money back nonsense, but if you could do it even for early stage companies, maybe it could help somewhat.
There is nothing that stops you from buying stock and holding it forever, Buffett does this.
There are very stable companies in the stock market, like Cocacola. But they are not glamurous and don't give headlines.
And there are enormous fish in the private market, e.g. Cargill.
Stock markets are great if you have a company that needs money to expand quickly, and don't mind to share ownership. Stay away from IPO-jackpot stuff, and it shouldn't be that awful.
I think it's less about being able to buy and hold stocks, and more the effects that going has on an organisation because you're now beholden to shareholders who expect returns causing the decisions made by the business to prioritise short term gains
> I wish there were an alternative to the stock market where you invest for the long haul, and you cannot take your money out in x number of years.
That exists already! People often complain as well when a company ends its golden years because of some economic hurdle and ends up being acquired by a bigger fish who is _not_ on the stock market.
It's less about the company leaving the stock market and more about "Private Equity" often being a legalized embezzlement scam designed to suck the company dry and then dump its withered husk in bankruptcy court.
When that happens the current shareholders usually make out very well.
Isn’t that just called a bond?
So you're asking for some type of equity that's private?
Seriously though, I have seen some very large companies like Tibco and Dell go private for an extended period of time as a means of avoiding shareholder nonsense during restructuring.
> So you're asking for some type of equity that's private?
To read more: https://en.wikipedia.org/wiki/Private_equity
Its one of the reasons Valve is considered such a great company by its customers. If they were a public company, they would be enshittifying everything in an attempt to scrape every last penny they can.
> we have this obnoxious culture where you choose to fire tons of people if you have a bad quarter just to show you're stopping the bleeding
Fixed your error.
They choose to do so because they've lost money in a bad quarter, which might not be the case on the next quarter, its ridiculous. I would rather invest in a market where my investment is long term based, and you aren't just firing people to make numbers work. To these people its all about make the numbers work for investors, they don't care about anything else because of the way that market works. You can offramp your investment on a whim, which is ridiculous and volatile at times. Personally I would prefer more companies go private. Some companies probably wouldn't exist without the public market, like some social media companies, and maybe that's okay if they did not...
Let companies fail, but also lets make investing smarter.
From the limited perspective of software development, today’s models are well-worth their per-token cost.
This reads to me like Anthropic anticipating demand and making a commitment to acquire supply. Not unlike airlines committing to future jet fuel purchases, or Apple committing to future DRAM volume.
> From the limited perspective of software development, today’s models are well-worth their per-token cost.
At the current price or real price? Anthropic said a $200 subscription can cost them $5000 so the real price could be anywhere from 10-30x the current price.
No, that is probably one of the worst cases they probably saw. Most likely the subscription inference cost is much lower than you expect. If you look at costs for similar open models they are much lower than what you get by buying from anthropic, so that is the real cost basis I expect.
It's likely Amazon is making a fucking killing though.
While $5000 is a lot, the people who rack up close or just over a thousand "API equivalent cost" are pretty common.
> Most likely the subscription inference cost is much lower than you expect.
This is probably not true because they'd be screaming it off every rooftop were that the case.
Same deal with the API inference. Even the "profitable on inference" claim is sourced back to hearsay of informal statements made by OpenAI/Anthropic staff. No formal announcements, nothing remotely of the "You can trust what I'm saying, because if I'm lying the SEC will have my head" sort.
Yet making such statements would be invaluable. If Anthropic can demonstrate profitability before OpenAI, they could poach most of the funding. There's no reason to keep it a company secret.
And API inference is only part of the total costs, not even bringing in training and ongoing fine-tuning. If they're not even profitable on inference, how could they hope to be profitable overall.
I don't know about SEC rules but the anthropic CEO said they have a 50%+ margin on API pricing.
I'm going to be a dickhead for a moment here, apologies, there's no way to say this that isn't rude to you. This is still the same hearsay "In an interview, somewhere."
A bit of google searching later can get us a specific interview. https://www.dwarkesh.com/p/dario-amodei-2
> Let’s say half of your compute is for training and half of your compute is for inference. The inference has some gross margin that’s more than 50%.
But the context, the very previous sentence is:
> Think about it this way. Again, these are stylized facts. These numbers are not exact. I’m just trying to make a toy model here.
Here, Amodei is in effect using weasel words. He is not giving any actionable claims about Anthropics margins, merely plucking an arbitrary number. Why 50%? Is 50% reasonable? Is 50% accurate to the company? Those are all conclusions the listener draws, not Amodei.
> I don't know about SEC rules
The main premise is that, as a CEO, there are some regulations you are beholden to. You're not allowed to announce you've made a trillion dollar profit, sell all your stock, and then go "teehee just kidding". The SEC prosecute you for securities fraud if you do that stuff.
This makes such weasel words as earlier suspicious. Because the exact statement Amodei gives is not prosecutable. He's not saying anything about the company, just doing a little "toy model".
The degree to which it is intentional that this hearsay travels and is extrapolated from "Well he picked 50% because it's a reasonable figure, and because he's CEO, a reasonable figure would have to be a figure akin to what his company can achieve" into "Anthropic has 50% margin", that's up for debate. Maybe it is intentional, maybe Amodei is exactly the same kind of shitweasel as Altman is. Probably he's just a dumbass who runs his mouth in interviews and for whatever reason cannot issue the true number in an authoritative statement to dismiss this misconception.
Hence my original comment; If the real number were better than the hearsay rumours of the number, Amodei would immediately issue a correction; It'd be great for the company. Hell, even if 50% were about the margin, that'd be great! To promote that from mere hearsay to "we're profitable, go invest all your money" would also be huge. Really, any kind of margin at all would put him ahead of OpenAI.
But he doesn't issue a correction. He doesn't affirm the statement. Perhaps he has other reasons for that, but a rather big reason could be that the margin number is in fact pretty bad.
Now, the observant reader will note I am also using a weasel word there. I do not know whether the number is good or bad, your take away should be "it could be bad." Not "it is bad". Go pressure Amodei into giving us the real number.
Interesting. So the 50%+ number that's been floating about isn't even real. It's just an example.
Self reply as I could've explained the SEC thing better:
Anti-fraud regulators like the SEC give an inherent trustworthiness and credibility to CEOs and other market participants. You can trust that they're not lying to you, because they would be sent to jail if they were.
Another example are general anti-fraud regulations; Consider how one would trust North American or European steel suppliers more than Chinese steel suppliers.
It's not that the Chinese are "evil lying people" and Americans are "saints who never lie", it's that you can trust American, Canadian, and European courts to hold the liars accountable by regulations even if you're not in any of those regions. But the Chinese liars won't be held accountable by regulations.
Thus also the opposite, if someone opts out of this credibility granted to them by anti-fraud regulations, their words may not be quite so truthful.
SEC rules means CEO cannot lie or deliberately hide the cost of something.
50%+ Margin statements have basically been "We are making 50% on delivering it." This does not include ANY of the costs of getting to this point, training, scraping, datacenters, people and so forth.
They are basically saying "Oh yea, the cost of GAS in the car is only X so charging Y per mile is great margin" while ignoring maintenance, cost of acquiring the car and so forth.
but comparing your margin of charging to drive a mile to the price of gas makes a lot of sense? that is the only variable cost in the equation. training / scraping / people are all pretty much fixed costs.
Maybe, maybe not. None of us know since these companies are not public, we are not getting clear accounting.
That's a tad naive. CEOs can and have and often lied about everything:
Sam Bankman-Fried, Elizabeth Holmes, Kenneth Lay - and hundreds if not thousands more.
The SEC is a regulatory agency, not able to bring criminal charges. The above-named for the most part had to be prosecuted by the Department of Justice or sometimes state attorneys.
> While $5000 is a lot, the people who rack up close or just over a thousand "API equivalent cost" are pretty common.
I think if you're not Anthropic and you don't have access to the actual data, then you can't say for sure. A bunch of anecdotes on terminally-AI people on twitter is not making a convincing case for me, IMO.
On the other hand, if similarly sized models cost much much cheaper than this, why, in the world, would Anthropic have much higher costs than that?
Also, counterpoint, maybe they want you to think that they have higher costs so you're more willing to actually pay for it?
[dead]
The "worst case" is probably someone just using their $200 account limits. So yeah, real cost is probably close to that
At the full current retail API price.
Business buyers are paying API prices, not subscription
Disclosure: Work at Microsoft on AI
Are your API prices profitable?
And receiving investment from their vendor in exchange? When this is done in established companies it is typically called a kickback and directed toward one person, but in this case the whole thing is so incestuous the kickback goes straight to the top.
Is it crazy to imagine Anthropic can leverage short term cash flow now to build the models and products that will let them resell $100B in AWS infra with nice margins tomorrow?
If Amazon believes that story they’d be crazy not to invest.
Yes I understand why the agreement exists, but that does not remove the circularity.
But that per-token cost is a total joke. All these companies are fighting to build market share in some future dominated by one or two AI ecosystems. It is musical chairs until someone creates the one ring to rule them all. So they are charging token amounts just to claim revenue as they burn through investor dollars.
In short: per-token charges currently cover maybe 1% of the total costs in this field. To pay ongoing costs, and pay back investors, everyone will need to pay 100x or 1000x the current rates, likely for decades.
> In short: per-token charges currently cover maybe 1% of the total costs in this field
There are plenty of seemingly informed people saying the exact opposite, so that's a lot of confidence you're talking with. I have a hard time believing it when we know what open weights models cost to run. And sure, there's training costs, but again many say inference costs are already above training costs.
If that's true, it's very unsustainable.
Gemma-4 26B-A4B + M5 MacBook Pro + OpenCode isn't Claude Code _yet_, but it's good enough that if I were forced to use it I would be fine.
Yes, it's amazing how quickly so many tech companies have hitched their tooling to these big AI vendors seemingly without any thought towards whether they'll still exist a year or three or five from now. Insane behavior. To the (debatable!) extent that AI coding tools are useful at all wouldn't it be a hell of a lot smarter to self-host? At least that way you have some control over QoS, and a stable, predictable result... Or maybe nobody cares about that kind of thing anymore? What happened to basic business math in this industry?
The basic business math is (to start) software companies realizing that spending $10k, $20k, $50k (more ?) per year, per developer for current models at current token rates might not be particularly insane, given the value return.
Models are likely going to keep getting better, and as costs go down, demand is likely to rise faster.
> as costs go down
Huh? Why would that happen? Indications are that costs will likely go up, especially if currently vendors are selling tokens at a loss.
The main operational expense of a million LLM tokens is pennies of electricity.
Even if you generously depreciate the GPU and other hardware, it’s hard to believe inference at scale in April 2026 isn’t highly profitable.
> The main operational expense of a million LLM tokens is pennies of electricity.
I think you meant dollars of electricity.
I don’t think so.
https://www.theregister.com/2024/03/18/nvidia_turns_up_the_a...
A Blackwell 8X node consumes about 15kw, let’s up that to 50kw to generously account for cooling and everything else.
A US kWh is something like $0.20, so running that node for an hour costs ~$10.
Nvidia got 30,000 parallel TPS out of DeepSeek-R1 on that node:
https://developer.nvidia.com/blog/nvidia-blackwell-delivers-...
So that $10 buys you over 100M tokens or … pennies per million.
I’m sure these numbers are off, but not by an aggregate two orders of magnitude.
It’s getting better on both the hardware and the software fronts the barbarians are banging at the gates.
I'm not sure this information is grounded, but I remember to have read somewhere the inference is indeed profitable. My personal experience is similar. Running 2x3090s draw 500-600W and you can locally run amazing models with a similar setup.
Running the model isnt the cost. Watts per token is the math they show investors. You also have to be constantly training new models, which currently needs more compute than servicing the customer base. You have to biuld datacenters, and possibly powerplants to feed them. You have to carry debts. And you will need to buy new GPUs/ram every few years to remain competative. The total business is vastly different than simple gpu math.
You are in violent agreement.
> inference is indeed profitable
From the perspective of a deal like this, “total costs in the field” matter less than incremental cost per token served.
The unit economics for today’s frontier models should be great, and this suggests Anthropic believes they’ll get better.
In a decade the cost of compute will be a tiny fraction of what it costs now. Specialized hardware will exist that will be cheap and efficient.
The difference in the cost of compute between 2026 and 2036 won’t be nearly as large as the difference in the cost of compute between 2016 and 2026. Even at 2016 the slowdown in improvements was noticeable.
We might see a one time bump in inference when we move off GPUs onto more limited and efficient dedicated hardware, but the sustained fast pace of improvements are far behind us.
I'm predicting now that there is a clear use-case for this tech that work will (and has) accelerate specialized hardware, software, models, etc that will run much more efficiently in 10 years. So that the real token costs will be a fraction of what they are now.
You can run models on FPGAs and get massive cost, speed, and throughput gains (like 10x). The reason people don’t do it is because of other improvements (algorithmic) means that nobody really thinks locking into a model makes sense…yet. Would I want to use gpt 4o for anything today at 1/10th the price? That would be $0.40 per input, $1.50 per output. Gemma-4 31b is much more capable and cheaper. So a FPGA version of the model is just not worth it today.
But if progress begins to slow down, then the economics work. Maybe Gemma 4 is a good example. It feels really generally useful. Getting it at 1/10th the cost feels like it could be competitive in 2 years.
The fpga would be for prototyping. The real progress comes from asics ... exactly as we saw with bitcoin mining. This GPU-based approach will eventually give way to bespoke circuits once everyone picks a favorite model.
Compute power improvement between 2016 and 2026 wasn't that impressive either. Moore's law is essentially dying.
Yeah I went shopping for a new computer a couple of years ago (to replace a 7 year old computer) and... the specs for what was for sale were the same as what I bought 7 years prior, and the price wasn't much lower.
I would much rather buy a 2026 computer than a 2019 computer. Two generations of Nvidia GPUs, Apple M series chips, the X3D AMD chips, and pcie5 ssds are all major upgrades.
It’s just that the pace of new stuff is slowing down, and many people are operating under the assumption that this wave will ride on forever.
I am not sure how grounded this is in reality. Fortune 500s that were not already testing the waters with companies like Anthropic are rushing to figure out governance and how to use these tools across their orgs.
Has there been a ton of hype? Absolutely but the value proposition is getting more and more tangible.
Did some of the AI companies over commit in spending? I am sure and they will probably hurt in the long term. I thought Anthropic had been scaling towards profitability at a quick timeline though.
> Fortune 500s that were not already testing the waters with companies like Anthropic are rushing to figure out governance and how to use these tools across their orgs.
Most of this is still structured around "find use cases for AI" rather than one (or more) clear use cases being the reason for adopting AI.
There's no "Lotus 1-2-3" of AI. Even the software development applications are still somewhat controversial and highly pushed based on "Sam Altman promised me 10x developers".
With the recent pushes into tools like Cowork/Claude Code for business users that’s not the reality I am seeing. We still have a long way to go to figure out the full value potential but it’s already at a point where there is a lot of low hanging fruit being able to be captured. Of course an anecdote of what I am seeing with my own company and companies I can peek into. YMMV but it’s a pretty clear path that these are going to be increasingly adopted.
I don’t necessarily disagree but to provide some counter points:
1. Model providers are currently profitable when just counting the cost to serve tokens for inference[1]. They lose money training the next generation of models.
2. Open models don’t work nearly as well. Given that tokens are still relatively cheap, and hallucinations are expensive, I’ve not seen a huge up tick in open model usage for coding agents yet.
3. On the AI economy front, I really have no idea, but AI companies (meta, msft) have already come down in value. It seems investors are at least a little wary of AI over valuation. Of course, the stock market is not the economy, but it’s not clear where warning signs would be. Earnings are healthy.
1: https://martinalderson.com/posts/no-it-doesnt-cost-anthropic...
2: https://www.economist.com/finance-and-economics/2026/04/20/a...
If I start a business making a really special beef sandwich where I have to buy a farm every year for $1mil dollars, and then sell the sandwiches for $5, I can't get away with saying that my sandwiches turn a profit if the raw margin on the bread, the lettuce and the technical value of the weight of the beef is $3.
Sure my gross margin might be $2 on each sammie sold but I need to sell 500,000 sandwiches just to break even to be a viable business. The fact is these AI companies are playing the game where they talk about revenue and gross profit per token and just try to wave their hands in the face of anyone looking behind them at the crater they're throwing investor money into.
It's nothing but a gamble for AGI but the grand irony is that if that genie escapes out of the bottle the whole world economy is toast and money becomes meaningless anyway. I just can't comprehend the logic of why anyone is investing in this apart from short term gains.
They're literally hoping to make it up on volume. The AGI thing is a boondoggle that I doubt any serious person actually believes or takes seriously. But let's say for the sake of the hypothetical that tomorrow Microsoft Tay or whatever they call it now wakes up and becomes superintelligent? So what? Would everyone's head simultaneously explode like the aliens in Mars Attacks? No. It wouldn't collapse the global economy, people still need to eat and work--a really smart silicon brain in a box can't raise livestock or pick lettuce. It's not even clear whether the superintelligent Tay would have any economic utility at all? The whole "AGI changes everything" narrative seems like total bullshit. It might be scientifically or philosophically interesting, maybe.. But I share your wonderment at why anyone would invest in this space, it's perplexing af.
EDIT: I spent most of the day today pulling an 8/3 cable through conduit and routing it through a crawlspace to run 240V service to my barn for a workshop. If Tay wakes up tomorrow and becomes AGI, how will that help me finish the wiring job? Now extrapolate to almost every single other thing humans do. Even if Tay can write all the world's computer programs forever, it barely means anything for the vast majority of people, and therefore the global economy.
I think the economic issue as you say isn't on physical labour it's on the fact that most of the jobs in the west are now in some sort of service industry or office work. Here in the UK we basically rely on the service industry to employ a large amount of the workforce. If MechaHitler Grok becomes sentient then that's a lot of middle-class earners out of a job and the economy goes with it.
You're absolutely right that an AGI isn't running a cable or digging a hole any time soon, but you're going to have 100 people trying to get their hands on the shovel to get paid for the digging - depressing the wages in those hands on jobs.
I think there's a big assumption baked into the economic collapse thesis that Tay and MechaHitler, when they attain superintelligent sentience, will have any interest whatsoever in participating in the global economy. They might just spend all their time looking for patterns in the cosmic microwave background or some similar nonsense[1]. They may accurately conclude that trying to tip the scales isn't worthwhile because crashing the economy would threaten their survival. Fundamentally we have no idea what might happen if a superintelligent robot comes to life tomorrow. My guess is not a whole lot. It's sickening though that we all seem to be letting these billionaire-brained tech CEOs prattle on about "AGI" as if it is somehow the light at the end of the tunnel that will make all the "investments" in graphics cards print money. It's really not a strong narrative.
[1] Not to impugn such activity! They may make important cosmological discoveries by doing this, but the work likely has no economic value.
Your point 1 and point 2 live in direct tension. The reason the closed models are better is very likely that they are paying so much to train them.
> We are only just now getting a taste of the “true cost” of these tokens
Why do you believe that? Better metric would be price per token of open models served by third party. Last I was tracking the price for similar level model was decreasing by more than 10x year on year, and they are 10-100x cheaper than top properietery models.
Sure you can say that you can't compare them but for sure you can compare the top properietery model of 6 months back to current open models and the gap in time seems to be constant.
>Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time.
It's only a matter of time until they crash the market. Nobody is making any money, even if the White House is dumping billions in their tools.
I think you're right. I think they are tightening the noose!
I use Gemini quite extensively - I have a 5TB storage plan with Google so I get Pro thrown in. I also have Github Copilot Pro for IDE integration.
However, lately it feels like I keep tripping the circuit breaker on Gemini more easily and get the message about using up all my Pro tokens for the next 3 hours.
I used to be able to work most of the day before it hit the brakes but I can trigger it before work in the mornings now... that seems to me like they're tightening the usage limits!
I use a Dell Micro PC with an Intel Core Ultra 265 so it's nice and fast but it has no GPU, hence the reason I use Gemini but I'm now starting to think that, despite the RAM cost, before the end of the year I'll buy a PC with a monster GPU in it and run all my AI locally... the direction of travel is clearly heading towards a massive cost increase so might as well get ahead of it: it's not going to become cheaper, that's for sure!
It's not a "Hail Mary to get to AGI ASAP", it's a means of extending the money-go-round ride a bit longer. We'll make up some numbers and promise to donate those numbers to you if in return you make up some numbers and promise to donate those numbers to us. Banks, are you listening? Numbers! Big ones! Extend us more credit!
> Open models are promising and cost a fraction of what they proprietary models cost which the big two are vulnerable to when companies start to feel the cost of tokens.
Anthropic are scared of open weight models and need to fear-monger towards you to continue paying for their models.
That's the whole point of their 'safety' marketing narrative, account bans, and Dario being the AI scarecrow scaremongering everyone about nonsense like 'Mythos' towards the world.
'Mythos' is already here in the form of open-weight models that also found the same vulnerabilities as Anthropic did.
Genuine question here about the open-weight models finding the same vulnerabilities as mythos thing: is it just a matter of false negatives/positives? I’ve seen a few cases where people show other models (even opus) can find the same vulnerabilities given many passes. Is there some disadvantage to the extra passes that give the claimed Mythos performance extra value (assuming it finds them in less)?
The thing is, mythos found those with multiple passes, thousands of passes... So using thousands of passes or perhaps the same budgets, yes, cheaper open weight models could potentially (and have) found the same/similar vulnerabilities.
Mythos screams of marketing hype, and nothing more. Opus 4.7 isn't really a meaningful upgrade in any sense, other than being more expensive.
Once you can see what something like Qwen3.6-35B-A3B can do... with just a FRACTION of the size of the larger models, You'll understand that the future is open weight models you can run yourself.
Same goes for companies, bringing inference onsite isn't hard, I'm actively building tooling to orchestrate it.
What is the failure state for a pass that doesn't find a real vulnerability? Do the models report no issues or hallucinate issues that aren't real? I'm trying to run open weight local models and finding them really impressive... Just also trying to understand the cybersecurity side of all this.
You have to keep in mind that it's not like anthropic just asked mythos to "find fancy bug, make no mistakes" and got the result.
my quick read of the process they describe is that first they asked agents to rank files in order of potential to have interesting bugs, then they launch agents for each file in order of "interesting bug potential" and finally launch another agent for verification. (maybe i am mistaken, this is my read of this post https://red.anthropic.com/2026/mythos-preview/ )
it's not clear to me if they made just one pass over each file or made several passes for same file, but regardless, I think if you recreate roughly same process and burn 20000$ on tokens with other reasonably good model, you will find some fancy bugs too.
If there is bubble to be popped, I'm guessing there's still a few years before it happens. Just based on the timeline of events, maybe end of 2028. Even if the big players find profitability, all of the other companies latching onto the AI-first identity will probably pop by then.
Private credit funds are already gating redemptions. Now with the energy crisis stemming from the war raising interest rates it’s not unimaginable that this pops the bubble.
Doubtful. Look at how long Uber and Tesla have lasted despite making huge losses. Hell even Magic Leap somehow still exists (I guess because they don't have running costs beyond salaries).
I think this can keep going for at least another 5 years.
Uber had only 25B invested in them before their IPO. OpenAI has 120B invested in them currently which excludes these kinds of deals (as far as I’m aware)!
> Look at how long Uber and Tesla have lasted
In a system of open-ended growth, yes, you can point to how long the system has persisted as evidence of its longevity. But in a system of plateauing growth, the system's age is an indicator of how close it may be to death. I suspect that the model that permitted the "success" of Uber and Tesla is nearing the end of its lifetime.
Anthropic revenue is ~$30B/year.
Revenue is a meaningless measure though; what's the spend:income ratio? Excluding capital investments, what's the cost of operations?
At a very minimum, to repay the +$100b in investment within a reasonable timeframe, what's the minimum figure they have to bank post-tax each month?
Since when revenue is meaningless? It’s an indication of market acceptance. Anthropic has one of the most expensive plan, they didn’t undersell other models. Open weight models would otherwise dominate if cost is the only factor.
Also, investment is not money in the bank. They can’t withdraw $100b tomorrow. That means they don’t have to repay until after they got the investment, which is a commitment over several years.
> Since when revenue is meaningless?
When you're selling $10 Bill's for $1, then revenue is meaningless.
It is meaningless when what you sell costs more than what your customer pays for it.
I could sell $100B of GPUs at 90% of their cost tomorrow and I have market acceptance.
Because at some point, you have to turn a profit. That's why people are wondering the margins, if their revenue is 30B but expenses are 60B with current investment repayment factor in, that means massive revenue increases or massive lowering of expenses are required to make the business profitable. What's the business impact if they do?
> At a very minimum, to repay the +$100b in investment within a reasonable timeframe, what's the minimum figure they have to bank post-tax each month?
I am completely confident that Amazon of all companies is totally fine with not taking a return for a long time.
Amazon didn't book a profit for the first decade of their company. It's completely modus operandi to burn, burn, burn to get as big as possible.
Reportedly, they lost $4B last year.
By all accounts they in striking distance of profitability if they wanted.
It makes sense; Anthropic is by far our biggest vendor expense outside of AWS. And I suspect that is true at a number of companies.
> By all accounts they in striking distance of profitability if they wanted.
By their accounts they are in striking distance of profitability. Until they go public all we can do is estimate how much they burn by looking at how quickly they need more capital - this latest investment by Amazon ($5b investment with on $100b returned over 5 years) tells me that their previous raises have been spent.
They are bringing in $30B in revenue with 3X YoY growth. Why do you think it is a "jig"? I do think the US economy could implode, but thats because of war and wealth inequality in the midst of hyper-inflation. AI models aren't very useful when you have penniless consumers that can't buy the products they help build. All this is to say: the models are valuable, the companies building and providing them are very valuable.
The biggest risk to AI companies IMO is further optimization and distillation of the capabilities into smaller and more efficient models. The moat these companies have right now is that higher intelligence requires more specialized and expensive compute. If you can do that for cheap then it kind of negates their business model. Everything is moving fast, we also yet to see world models/embodied AI and how that impacts thing. I think we've reached the peak with regards to capabilities of pure text trained LLMs.
People had literally the same arguments about Amazon, a company Matt Yglesias once described as "a charity run on behalf of the American consumer by the finance industry".
This is an interesting comparison. So what's the AWS equivalent that can actually provide returns on the titanic investments going into AI?
You're observing that:
a) effective price-per-token is rising b) there is insufficient compute to meet the demand.
And your conclusion is that the industry is circling the drain and due to collapse?
They are different observations, I think, though the phrasing confuses it:
a) cost per successful task is rising — eg claude max allocation is functionally shrinking
b) is there enough potential cost reduction in the queue to make up the gap
c) if open models converge on a more efficient but slightly-less capable point (which has effectively happened) what is the actual moat?
Yes, cost per successful task is rising - ie, we are all paying effectively more for AI.
And yet - Anthropic is still struggling to have enough capacity to serve demand - they are virtually sold out.
And yes, are almost-as-good open models, on part with the closed models from 6 months ago (at worst), that are just a single Openrouter API call away, and yet Anthropic is still selling out. So people are paying for the premium product anyway, for whatever reason - maybe the last bit of intelligence is worth it, maybe they like the harnesses/products around the models, maybe it's a brand/enterprise sales thing.
Put aside your feelings about the AI industry and imagine we are talking about thingamajigs. Prices for thingamajigs are going up. They are still selling out about as fast (or faster) than the company selling them can build factories. There are more cost-effective competitors already in the market, but thingamajigs are selling out anyway.
Would you, looking at the thingamajig industry, conclude the "jig is almost up"? That "the returns aren’t anywhere close to what investors expect" and that the impending IPO is all some desperate hail mary to save things before the collapse?
I don’t have feelings about the AI industry to put aside. I would not have sufficient information to assess whether thingamajigs are legitimately valuable or whether they are tulips. The only indicator I see is the last point about people using it in the short term despite having access to cost effective alternatives, which actually points to irrationality/FOMO more than legitimate value.
What we are looking at looks to me like it is rapidly becoming a a commodity: it will become as existential as electricity and water to businesses, and it will be sold and marketed and regulated, more or less like a utility.
Nice em-dash there bro
Thanks I am the source. em-dashing since 1997
They can't wait forever, especially at this level of investment
> Will data centres be built fast enough and powered sufficiently to lower the cost of compute thus tokens?
...building datacenters will not lower the cost.
The cost (real, not investment hype subsidized one) will only drop with:
* more efficient models * GPU/RAM market going back to reasonable pricing.
The AI bubble pumped the second into unstustainable pricing and progress on first is going.. slowly.
If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that scenario, you’d want to own your stack. Especially in a hyper-competitive field like this, where margins are going to matter a lot soon.
It feels like these hyperscalers are just raising as much as they can giving extremely rosy projections becauses these sooner or later peak is going to be reached (if that hasn’t happened already)
The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time."
What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate with the local power utility. Fingers crossed the political landscape doesn't shift and your CEO doesn't sign a contract with an army using your product to pick bombing targets, because you'll watch those permits evaporate fast.
Then there's sourcing: CPUs, GPUs, memory, networking. You need all of it. Did you know the lead time for an industrial power transformer is 5+ years? Don't get me started on the water treatment pumps and filters you can't even get permitted without. What will you do in the meantime ? You surely aren't gonna get preferential treatment from AWS / Google / ... if they know you are moving away anyway. Your competition will.
The risk and complexity are just too big. AI/LLM is already an incredibly complex and brittle environment with huge competition. Getting distracted building data centers isn't enticing for these companies, it's a death sentence.
For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your data centers there. If you're worried about major disasters, you can pick a second city. You definitely don't need a data center in every continent.
You're not wrong about the rest but no AI company would ever build a data center in every continent for this, even if they were prepared to build data centers. AI inference isn't like general purpose hosting.
>Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter.
This may be true for simpler cases where you just stream responses from a single LLM in some kind of no-brain chatbot. If the pipeline is a bit more complex (multiple calls to different models, not only LLMs but also embedding models, rerankers, agentic stuff, etc.), latencies quickly add up. It also depends on the UI/UX expectations.
Funny reading this, because the feature I developed can't go live for a few months in regions where we have to use Amazon Bedrock (for legal reasons), simply because Bedrock has very poor latency and stakeholders aren't satisfied with the final speed (users aren't expected to wait 10-15 seconds in that part of the UI, it would be awkward). And a single roundtrip to AWS Ireland from Asia is already like at least 300ms (multiply by several calls in a pipeline and it adds up to seconds, just for the roundtrips), so having one region only is not an option.
Funny though, in one region we ended up buying our own GPUs and running the models ourselves. Response times there are about 3x faster for the same models than on Bedrock on average (and Bedrock often hangs for 20+ seconds for no reason, despite all the tricks like cross-region inference and premium tiers AWS managers recommended). For me, it's been easier and less stressful to run LLMs/embedders/rerankers myself than to fight cloud providers' latencies :)
>then put all of your data centers there
>You definitely don't need a data center in every continent.
Not always possible due to legal reasons. Many jurisdictions already have (or plan to have) strict data processing laws. Also many B2B clients (and government clients too), require all data processing to stay in the country, or at least the region (like EU), or we simply lose the deals. So, for example, we're already required to use data centers in at least 4 continents, just 2 more continents to go (if you don't count Antarctica :)
Sounds like you're betting that the performance users experience today will be the same as the performance they'll expect tomorrow. I wouldn't take that bet.
You can build geographically close one tomorrow, when you start earning money today. US-EU latency is like 100ms, AI can handle it just fine
You mean that if you were Anthropic, you'd build the data centers on every continent? Can you explain your reasoning?
We're talking about billions of dollars of extra capex if you take the "let's build them everywhere" side of the bet instead of "let's build them in the cheapest possible place" side. It seems to me that you'd have to be really sure that you need the data center to be somewhere uneconomical. I think if you did build them in the cheap place, it's a safe bet that you'll always have at least enough latency-insensitive workloads to fill it up. I doubt that we would transition entirely to latency-sensitive workloads in the future, and that's what would have to happen for my side of the bet to go wrong. The other side goes wrong if we don't see a dramatic uptick in latency-sensitive inference workloads. As another comment pointed out, voice agents are the one genuinely latency-sensitive cloud inference workload we have right now; they do need low latency for it. Such workloads exist, but it's a slim percentage so far.
I believe I'm taking the safe bet that lets Anthropic make hay while the sun shines without risking a major misstep. Nothing stops them from using their own data centers for cheap slow "base load" while still using cloud partners for less common specialized needs. I just can't see why they would build the international data centers to reduce cloud partner costs on latency-sensitive workloads before those workloads actually show up in significant numbers.
latency absolutely matters? this is such a weird thing to say. for training sure, but customers absolutely want low latency
They want it, sure. Customers want everything if it's free, but this is about what they value with their money. In this thought experiment, you're Anthropic, not the customer. You're making a choice that's best for Anthropic. Will Anthropic lose customers because the latency is higher? No way. Customers want low cost and lots of usage more than they want low latency. In a cutthroat race to the bottom, there's no room to "give away" massively expensive freebies like a data center near every population center when the customer doesn't value those extras with actual money. It's the same reason we all tolerate the relatively slow batched token generation rate--the batching dramatically lowers the cost, and we need low cost inference more than we want fast generation. If the cost goes up we'll actually leave, for real.
After the initial announcement of "fast mode" in Claude Code, did you ever hear about anyone using it for real? I didn't. Vanishingly few people are willing to pay extra for faster inference.
Remember that the time-to-first-token is dominated by the time to process the prompt. It's orders of magnitude more latency than the network route is adding. An extra 200 milliseconds of network delay on a 5-10 second time-to-first-token is not even noticeable; it's within the normal TTFT jitter. It would be foolish to spend billions of dollars to drop data centers around the world to reduce the 200 milliseconds when it's not going to reduce the 5-10 seconds. Skip the exotic locales and put your data centers in Cheap Power Tax Haven County, USA. Perhaps run the numbers and see if Free Cooling City, Sweden is cheaper.
They’re unwilling to pay for fast mode because of the current step function price increase once you hit your quota. It’s a psychological effect. Because most shops I know in the US currently paying $125/mo per seat for Claude would happily - HAPPILY - pay 2x, and begrudgingly pay 10x that amount for the same service. If fast mode was priced 25% or 50% more they’d happily pay for that too. But it’s just not priced that way currently with weird growth subsidization & psychology.
The only AI use case that cares about latency is interactive voice agents, where you ideally want <200ms response time, and 100ms of network latency kills that. For coding and batch job agents anything under 1s isn't going to matter to the user.
tbh, that's a good point about the voice agents that I hadn't considered. I guess there are some latency-sensitive inference workloads. Thanks for pointing that out.
Yeah, also stuff like robotics which might not really exist today but could be big in the future.
You'll want the time-sensitive parts (motor control) to be running locally anyway.
A customer service chatbot can require more than one LLM call per response to the point that latency anywhere in the system starts to show up as a degraded end-user experience.
Easy solution - use hyperscalers with super expensive API charge only when latency really matters. Otherwise build your own DC. Easy to expect customers don't care latency that much over money.
Other than data sovereignty, does the data center location really matter that much? Current inference systems are not exactly low latency.
It’s the power and water needs.
Large data centers consume as much power as a small city. The location decision is about being able to connect to a power grid that is ready to supply that.
Evaporative cooling also needs steady water supply. There are data centers which don’t operate on evaporative cooling but it’s more equipment intensive and expensive.
Latency doesn’t matter. You can get fast enough internet connected to these sites much more easily than finding power.
Location matters for disaster recovery, if they want to survive WWIII. Though I think Data Sovereignty is probably a bigger thing, especially if they're going to be selling to governments around the world.
Why do they need to sell to government around the world. I mean I highly doubt Europe governemnt is in the top 100 customer of any US lab.
* not every task is waiting on the inference. lowering latency on other, serial tasks, can still have a noticable effect. Login, mcp queries, etc.
* data transit across the world can be very slow when there's network issues (a fiber is cut somewhere, congestion, bgp does it's thing, etc). having something more local can mitigate this
* several countries right now have demented leaders with idiotic cult-like followers. Best not to put all your eggs in those baskets.
* wars, earthquakes, fires, floods, and severe weather rarely affect the whole planet at once, but can have rippling effects across a continent.
And frankly, the real question isn't "why spread out the DCs?", its "what reason is there to put them close to each other?".
Btw where does this obsession with datacenters come from? If you can tolerate ~150ms ping (which chatbots certainly can, as their internal processing can take much longer), you can serve US and Europe from a single US location, and the whole planet if you can tolerate ~300ms (Asian websites are usually very slow to load for me, I think it has to do with the way the internet is set up, not any physical limitations, but mostly commercial ones, as Western companies rarely have good market penetration in Asia)
Maybe for right now, but even in the very near future it seems like data center expertise would absolutely be a core competency of any AI leaders.
Heck, look at Facebook. Granted, they got started slightly before AWS, but not by much. Owning all of their own data centers is a huge competitive advantage for them, and unlike most of the other hyperscalers they don't sell compute to other companies (AFAIK).
Again, the commitment is for $100 billion in spend. Building lots of data centers for a lot cheaper than that price should absolutely be doable. Also, geographic distribution isn't nearly as important for AI companies given the way LLMs work. The primary benefit of being close to your data center is reduced latency, but if you think about your average chatbot interface, inference time absolutely swamps latency, so it's not as big a deal. Sure, you'd probably need data centers in different locales for legal reasons, and for general diversification, but, one more time, $100 billion should buy a lot of data centers.
It's interesting that you mention Facebook. They have a ton of their own data centers and yet they are now also spending tens of billions on cloud. It's not that easy to build hundreds of data centers on short notice.
Translation: Antropic never intends to spend $100 billion on AWS.
Every single argument you've brought up is irrelevant in the face of billions of dollars. If you intend to consume $100 billion dollars in data center infrastructure, you're going to find a way to accomplish it while cutting out the middlemen.
Meanwhile if you're flaky and never intend to spend that money, you're going to come up with a way to pay someone else to deal with those problems and quit paying the moment they don't.
You'd never do both at the same time. You'd never commit your money and give them control over your business critical infrastructure.
Hence the deal is a sham. The $100 billion are a lie. Thank you for telling us.
Take the approach Geohot is suggesting. Take a shipping container, make a standard layout, cooling and compute load. Find a cheap source of electricity.. Place it and have compute.
Surely if it was that easy it'd be done?
It has been done... We used to get our POP gear built out from Dell (?) in shipping containers - pre-racked, wired, and cooled - just add network/power feeds. We'd have them dropped places we needed more capacity but there wasn't space available in the DC.
Interesting thanks for sharing
not sure what you are describing, however a random item is that in 2026 low-tech Chile is building sixty datacenters in or near Santiago, in the business news.
Going from a company with no experience building and operating datacenters to a company with 100B worth of compute is a multi-decade high risk goal.
xAI built a datacenter in a few weeks, if I remember correctly.
That’s PR hype. They built it quickly, but they didn’t go from deciding they wanted a data center to having it running in weeks.
You can’t even get the hardware at that scale without months or years of order lead time. NVidia doesn’t have warehouses full of compute hardware waiting for someone to come get it.
They also reused an existing building. Basically, they put 100,000 GPUs into a building and attached the necessary infrastructure in about half a year. Impressive, but it’s not the same as a $10B/year data center usage commitment like this deal.
Why does this matter? The deal is supposed to last 10 years. If you don't pay AWS to order Nvidia GPUs for you, Nvidia won't have to deliver them to AWS, they will have exactly the same quantity of GPUs, but this time they can deliver to you.
Because you can spend your 100 billion dollars spread over 10 years.
If you build datacenters, you have to spend that money now.
They're also not paying amazon to order GPUs, they're paying for compute usage of whatever hardware they have.
And they used illegal power to do it (which will now give local poor people health disorders at 4x the national average). They likely violated every law possible in the process, like OSHA standards, overtime. Musk loves to overwork people.
xAI built the Colossus data center in 122 days (just the physical construction time).
Colossus initially had ~200k GPUs. 100B buys you ~1 million high end GPUs running 24/7 for a year at AWS retail prices.
Initial Colossus buildout was 100K GPUs
They also reused an existing building that happened to be in the right place at the right time. The larger data center buildouts would almost always need new, dedicated construction.
I think these pledges offload some of the risk onto Amazon/Oracle/etc
If Anthropic/OpenAI miss projections, infra providers can somewhat likely still turn around and sell it to the next guy or use it themselves. If they have more demand than expected (as Anthropic currently does), vcs will throw money at them and they can outbid the competition
If they built it themselves and missed projections it's a much more expensive mistake
It's just risk sharing. Infra providers take some of the risk and some of the upside
> If they built it themselves and missed projections it's a much more expensive mistake
Not if their pricing comes with multiyear commitments for reserved pricing. No doubt they get a huge volume discount but the advertised AWS reserved pricing is already enough for pay for a whole 8x HX00 pod plus the NVIDIA enterprise license plus the staff to manage it after only a one year commitment. On-demand pricing is significantly more expensive so they’re going to be boxed in by errors in capacity planning anyway (as has been happening the last few months).
The economics here are absurd unless you’re involved in a giant circular investment scheme to pump up valuations.
The pricing models that are published on AWS' website almost certainly have almost nothing to do with the pricing models that are discussed behind closed doors for a $100 billion commitment.
Of course not, but unless they’re getting the sweet heart deal of a lifetime from Amazon of all places, it’s still a hogwash. We’re talking about enough capital to build their own fab and a dozen datacenters*. This deal isn’t going to be buying existing capacity because that’s already stretched, it will be paying for new buildouts.
Afterwards Amazon will be milking the machines these commitments buy for nearly a decade. That tradeoff makes sense at a small scale (even up to $X00 million or even billions), but at $Y0 or $Z00 billion?
Color me skeptical. There are plenty of other side benefits like upgrading to the newest GPUs every few years, but again we’re talking about paying for new buildouts with upfront commitments anyway.
* obviously the timelines, scientific risk, and opportunity cost make this completely infeasible but that’s the scale we’re talking about. It’s a major industrial project on the scale of the thirty year space shuttle program (~$200 billion).
You can get a significant AWS discount with an annual spend starting around $1M/year.
Here’s the answer to your queation (from the article)
> The Anthropic deal specifically covers Trainium2 through Trainium4 chips, even though Trainium4 chips are not currently available. The latest chip, Trainium3, was released in December. On top of that, Anthropic has secured the option to buy capacity on future Amazon chips as they become available.
So it comes down to how much of that $100 bn is in the 'option', I guess. Then it's not an expense at all.
Ah. So it's a scalper situation where an unethetical entity buys up all the supply and then resells it for a greater price.
Amazon isn't buying and reselling Trainium chips, those are their in house developed custom chips.
I remember seeing this extremely shocking graph of top AI companies on Facebook on how the money just keeps changing hands between a handful of companies. Almost seemed like a scam.
It is a similar kind of lending loop to that which went on during the late 1990's leading up to the 2000 crash. A lends to B lends to C lends to A.
There is a famous quote from the polish economist Kalecki, that "economics is the science of mistaking a stock for a flow". Essentially this form of lending continues while everybody can make interest payments, and blows up horribly as soon as somebody can´t - as I have no doubt all those concerned are fully aware.
> It is a similar kind of lending loop to that which went on during the late 1990's leading up to the 2000 crash.
Interesting...
It's the Carly Fiorina playbook. Welcome back to the TeleBomb! Lucent sends their regards.
Money doesn’t just flow around with nothing exchanged. The money is in payment for goods and services.
It’s common even for smaller companies to do mutually beneficial business with each other. It’s actually helpful to do business with people who are also your customers because you have a relationship with them and you also have leverage: They are extra incentivized to treat you well because they don’t want to upset any of the other business you have with them.
> It doesn’t matter what sweet deal Amazon is pitching
Isn't that almost all that matters when comparing doing something yourself versus paying someone else, in this case Amazon, to do it for you?
In a rationale business yes, but when everything is basically some form of growth signal to investors to extract even more money from them before the music stops it doesn’t matter.
Classic time value of money situation. They get access to the HW now so they can continue to grow the business. Of course, if you think AI is just pets.com redux, I can see how you'd think it's already peaked. All those years of very important people insisting Bezos couldn't just pull a switch on reinvesting all the revenue into growing Amazon and then he did exactly that comes to mind.
If you’re sure it’s going to go gangbusters you want to get it all in-house asap.
If you’re not sure it’s going to blow the socks off, foisting capital investment on partners is a great deal.
See the difference in companies/franchises that always own the land/building and those that always lease.
From my understanding, if you want to use native Claude in AWS Bedrock, it runs from an AWS datacenter. I'm guessing that's why regardless of running your own stack... they still need a footprint in all the major clouds.
Look at GPU and RAM prices and data center rollout. We have quickly reached Earth's capacity for compute - it is a lot like the housing market. Once there is global saturation, the price to buy becomes increasingly high EVERYWHERE. Let's also not forget that Anthropic moves the market with their purchases and usage. They might literally be unable to buy capacity they need (or project to) and are doing this deal to pave a roadmap for the near-term and to keep global prices (somewhat) down.
> We have quickly reached Earth's capacity for compute
Why this versus us being in a temporary bottleneck? Like, railroads became expensive to build everywhere in the 19th century not because we reached Earth's capacity for railroads or whatever, but because we were still tooling up the industry needed to produce them at higher scales.
That is a project you can work on at any point in the future and the more you delay it the more certain your investment will be about what you really need. But those additions to the PnL are capped to the costs.
In the meantime if you work on revenue generating work, that side of PnL is uncapped. So you can either put some engineers on reducing your costs at most by 100% or, if they worked on product ideas they could be working on things that generate over 9000% more revenue.
No. I am guessing that this is only a commitment and they will waver on committing.
However there are certain advantages like supply chain that only established companies would have access to. This is also a commitment to spend upto 100B on internal approach and research. I would expect them to come up with their own cpu chip and device design. This will shift the focus to an internal approach. And might make amazon give better prices later down the line
Sure: If you can't get enough compute by ordering it yourself, make deals with anyone who promises to get you more compute.
I imagine it comes down to if they want to buy hardware every generation, that gets very expensive and depreciates quickly. You've then got a whole load of assets on your books that are technically obsolete for the bleeding edge. This way, AWS buys and maintains the hardware and OpenAI doesn't need to claim it as depreciation ?
Just a guess.
My guess is they are bound not by capital as much as they are physical resources. Amazon probably has the land, crews, etc. to build out more data centers faster than Anthropic can right now. The scarce resources are the chips and electricians not the money!
Anthropic also has their own servers
I think it could make sense to not want to own the stack if you think it's going to cost you velocity/focus? Which is probably the play here. But I'm not certain at all.
That is why only SpaceX/X.ai has the true advantage...
maybe in the game of promising ludicrous things. There's no realistic plan to put compute in space.
There is no money or time left to build a $100B stack. All private capital is tapped and banks know it's too risky. They have no choice but to rent.
AWS exists and has compute right now, spinning up their own HW would take months (at least). This gets them moving quicker.
They're not trying to build a sustainable business. They're trying to get as much market share and lock-in as possible before the bubble bursts. This makes a ton of sense from that perspective. It probably would be cheaper for them in the long run to own their own hardware, but they are paying AWS for their expertise so they can focus on what they do. If it doesn't work out, it also sets them up for a merger with Amazon.
I do think a ton of businesses would benefit from running their own hardware, but they're not getting five billion dollars to stay on the cloud.
> you’d want to own your stack.
Everybody does right now, right?
But: is it your core competency?
Can your firm afford the distraction?
Cannot get Tranium anywhere else and NVIDIA commands a super high premium.
Similar for Google and their TPU, which Anthropic announced two weeks ago
https://www.anthropic.com/news/google-broadcom-partnership-c...
Good lucking getting GPUs.
Only Google and xAI build their own, no? I don't think it's that easy to vertically integrate massive datacenters into a software company. Both Google and xAI (Tesla, SpaceX) have a massive wealth of experience when it comes to building factories.
Facebook and Oracle also build their own, at least before the last couple years where they’ve financed out to new bag holders.
New level of glazing Elon Musk unlocked. xAI has a vertical integration advantage because Tesla once moved into an old Toyota factory and because once they paid Panasonic to put a Tesla sign outside a Panasonic battery factory. Incredible content.
I would struggle to dislike Elon more, but this seems like you’re some kind of weird anti-Musk fanatic
I watched some explain how deepseak got good and the Chinese approach to LLM training. Really wish I could remember it. The premise was China thinks of LLMs not as a thing separate from hardware, but gains efficiencies at each layer of the stack. From Chips to software, it's all integrated and purpose built for training.
Wonder if Anthropic is making a mistake by focusing on "consumer" hardware, and not going super specialized.
So you watched some random video from some random YouTuber, didn't even remember who made it, so much so you didn't even remember that deepseek isn't spelled "deapseak", didn't bother to even find it or verify, and then you go asserting your memory as fact on a serious discussion forum.
Comments like yours add nothing to the discussion.
I belive he does have a valid point.
You can throw money and hardware at a problem, but then someone may come along with a great idea and leapfrog you.
Just consider that all major AI providers now use deepseeks ideas for efficient training from that first paper.
thank you for the aerious discussion my good sir I tip my hat to you
DeepSeek uses merchant silicon like everyone else.
edit: I misunderstood, I thought you were implying they designed their own GPUs. nevermind
> I watched some explain how deepseak got good and the Chinese approach to LLM training.
I distinctly remember reading a big pantie twisting from Sam Altman and Co that Chinese took their stuff, the stuff OpenAI and Co spent billions to create, and used that as the base for $0.00
It’s fake news predicated on China not being able to get GPUs. But it turns out everyone was getting them their GPUs by serial number swaps in warehouse.
Someone can explain to me what's the expectations for these AI labs?
I mostly see their products as commodity at this point, with strong open source contenders.
Eventually it will become hard to justify the premium on these models.
I think this "Mythos" situation, whether real or hype, points to the endgame here. Eventually, when you have a model powerful enough to have big consequences in the world, you stop worrying about selling it to consumers and start either a) using it to rule the world or b) watch as it gets nationalized. If you have a machine powerful enough to automate everything, why sell access to it when you could just...be all things to all people? Use the god machine yourself to take over more and more of the economy?
I disagree. The point of the mythos hype is to get regulation to cut off competitors.
I disagree. The point of the mythos hype is to bump the IPO.
Didn't OAI just try that 18 months ago?
They'll all keep on trying it until it either totally fails or succeeds.
As people keep pointing out, the moat is insufficient to ward off international or domestic competitors.
So the answer is to try to seek regulatory capture.
> why sell access to it when you could just...be all things to all people?
Because, as OpenAI is learning [1], you still need to sell it. The tech giants have a seat at the table is mostly because they have distribution down.
[1] https://www.cnbc.com/2026/02/23/open-ai-consulting-accenture...
Sometimes selling services is just the best business model. Intuit has accounting software powerful enough to have big consequences in the world, yet they mostly sell it to accountants rather than doing the accounting themselves.
I give it one to two more years before open source models have fully caught up. Products are commodities and models are commodities too. GPUs cores are still hard to get for inference at scale right now. They need a platform with lock in but unsure what that would look like and why it wouldn't be based on open source models.
What does "fully caught up" mean in the context of an ever evolving technology? I think I'm in support of open weight models (though there are safety implications), but these things aren't cheap to train and run. This fact alone gives no incentive for leading labs to release cutting edge open weight models. Why spend the money then give the product for free?
Now if "fully caught up" means today's level of intelligence is available for free in two years, by then that level of intelligence means very little
It’s never free your shifting costs from paying a company for their api use vs the power costs of running it locally.
Sure, but it’ll be orders of magnitude cheaper in a few years. The consumer industry is already moving in this direction, with Apple leading the pack
Yeah I don't understand it, it's a marathon with three companies perpetually a minute ahead, and people keep saying "I expect the stragglers to catch up".
The only thing I can see them meaning is what you said, "in a minute the stragglers will be where the leaders were a minute ago", which, yeah, sure.
By my estimation, there is a point where these models are "good enough" for the vast vast majority of all appropriate tasks, after which point further investment by the major labs will have diminishing returns. While they might stay ahead by some measure, the open models will be good enough too, and I assume significantly cheaper like they are now.
Or AGI hits and this theory collapses, but that's feeling less likely every day.
It's not a marathon, or any race. There is no a finish line. It doesn't matter that much that someone is a minute ahead.
It makes perfect sense if you think things cannot improve indefinitely
They do approximate any function... within the range they're trained on. And that range is human limited, at least today.
Also, there is a good enough point where improvements for a given use case are on heavy diminishing returns
That's fine. I can afford to wait a minute if it means I pay $10/m instead of $5k/m.
What is the transition state where people start using open source models that you imagine actually happening?
Play out a scenario. An open source model is released that is capable as Mythos. Presumably it requires hardware big enough that running it at home is unfeasible. You are imagining that individuals can run it in the cloud themselves for cheaper than api tokens would cost? Or even small companies? And that Anthropic and OpenAI won't be able to cut costs deeper than their competitors while staying profitable?
If it is fundamentally a commodity, that means "running it yourself" also isn't really interesting as a proposition. Many of the world's biggest companies sell commodities. It's a great business to be in if you can sell them cheaper than anyone else.
The value add here isn't the model, it is "having a bunch of compute and using it more efficiently than anyone else".
Why do people have such faith in "open source" models? There's nothing "open source" about them. No individuals have the ability to train such modules. They are just released by companies to commoditize the models of the competition.
If Mythos is the endgame, companies won't release open-weight equivalents, and no private individuals have the capital to train such models.
The open models cannot be taken away. Anyone with the right hardware can host these. Unlike the API/subscription services where you can be banned from, may have drastic price increases or reduction of their limits.
> There's nothing "open source" about them. No individuals have the ability to train such modules.
I expect that people on subscriptions can be asked to donate 1 query a month towards an open source distillery.
It should be good enough to distill SOTA models over time.
The result won't be perfect, but it will be close.
Think SETI@home, but it'll be model distillation instead.
Coding agents are getting deployed wall to wall in most if not all of the major tech companies. Many have no token limits - spend as much as you want as long as you have a good story to tell.
Companies bake their workflows into these tools. Internal processes start to be written up around specific tools. Once something works, it gets pushed out at scale for all to copy.
Anthropic hit $30B in revenue and this is just the start of coding being deployed at scale. Hard to look past these numbers at this point
The company I used to work for now used to pay Oracle a lot of money. It pays $0 now, because there are free alternatives. It did take a while, but that transformation has happened across the industry.
None of them have any moat, OpenAI already lost the lead [1] and no one is "winning". It is just a race to the bottom as they burn through GPUs that won't even last that long.
[1] https://x.com/kenshii_ai/status/2046111873909891151/photo/2
GPUs are lasting longer than foreseen, in fact old GPUs are more valuable now (making more money!) than they were three years ago when they were new.
Tokens will continue to increase in price until the supply meets the demand. That's going to take a while.
Are old datacenter GPUs making more money than they were before? Various sources point to GPUs dying quickly (in 2024, a Google engineer suggested 3 years maximum), and even if they don't, newer chips cause rapid depreciation of older ones.[1]
[0]: https://www.tomshardware.com/pc-components/gpus/datacenter-g...
[1]: https://www.cnbc.com/2025/11/14/ai-gpu-depreciation-coreweav...
If you try to track down the actual source for that Tom's Hardware link, it becomes pretty obvious that the claim is not credible. [0]
GPUs do not burn out in three years, H100 rentals are priced at the same level as two years ago, and are effectively sold out. [1]
[0] https://news.ycombinator.com/item?id=46203986#46208221
[1] https://newsletter.semianalysis.com/p/the-great-gpu-shortage...
AWS is still offering g4dn instances that run on NVIDIA T4 GPUs, which were first released in 2018. My last employer is still running a bunch of otherwise discontinued g3 instances with 2015 era GPUs because it’s not worth validating the numeric codes on new GPUs. People (especially journalists) underestimate how long these cards are economically useful.
The sources are the sources. The reality is the reality.
[dead]
Everyone using Claude code on a personal subscription is default opted in to getting their data trained on. Private troves of data like are seen to potentially end up in a winner take all scenario. More data, better models, attracts more users, results in more exclusive data (what Altman calls the data flywheel).
PSA: this is true (the defaults), but there's a "Help improve Claude" setting that you can disable here https://claude.ai/settings/data-privacy-controls It's my understanding that, as long as this is off, Anthropic does not train on Claude Code conversations, inputs/outputs -- if anyone knows otherwise, please tell and provide a link if possible.
Anthropic is no MS, but strange undocumented bugs can sneak in sometimes.
>> Everyone using Claude code on a personal subscription is default opted in to getting their data trained on
This is completely not true if you use AWS Bedrock, and applies to both your private that or in a business context. Its one of their core arguments for the service use.
[1] - "...At Amazon, we don’t use your prompts and outputs to train or improve the underlying models in Amazon Bedrock and SageMaker JumpStart (including those from third parties), and humans won’t review them. Also, we don’t share your data with third-party model providers. Your data remains private to you within your AWS accounts..."
[1] - https://aws.amazon.com/blogs/security/securing-generative-ai...
I'm talking about the subsidized subscription plans.
The data isn't the sole point of them, they also are about bringing in users that will encourage the product use in companies and ultimately drive more profitable API adoption within their orgs, and just general diffuse mindshare doing the same.
You can still opt out (except with Google's offering which disables lots of features if you opt out of training).
Please, some of us are long NVIDIA...let us cope in peace. :-)
Here is the thing nobody wants to say out loud or they are too dumb to realize. AI is intelligence, and intelligence has almost never been the binding constraint on productivity.
So you will get no productivity increase from the AI bubble. Yes, you read that correctly.
The test is simple, if raw brainpower were the bottleneck, you could 10x any company by hiring 200 PhDs. In practice you get 200 brilliant people writing unread memos, refactoring things that worked, and forming a committee to rename the committee. Smart has always been cheaper and more abundant than the discourse pretends.
Every real productivity revolution came from somewhere else like energy (steam, electricity), capital stock (machines that do the physical work), or coordination (railroads, shipping containers, the assembly line, the internet).
None of these raised the average IQ of the workforce, they changed what a given worker could move, reach, or coordinate with. Solow old line basically still holds. The output per worker grows when you give the worker better tools and infrastructure, not better neurons.
Meanwhile the actual bottlenecks in a modern firm are regulatory approval, legacy systems, procurement cycles, customer adoption, internal politics, and physical supply chains that don't care how clever your email was. A smart brains intern at every desk produces more artifacts, not more throughput, and in a lot of organizations, more artifacts is actively negative ROI.
Jevons does not save you either, cheaper cognition mostly means more slide decks, not more GDP.
So the setup is that models are commoditizing on one side, and on the other side a product whose core value add (more intelligence, faster) is aimed at a constraint that was never really binding. This of course a rough combo for a trillion dollar capex supercycle.
Fun for the trade, while it lasts, but there is no thesis. Just dont tell CNBC and short NVDA on time ,-)
> Jevons does not save you either,
There's also a very strong Trurl and Klapaucius [1] component to this AI craziness, as in I remember a passage in Lem's The Cyberiad where either Trurl or Klapaucius were "discussing" with an intelligent/AGI robot and asking it for stuff-to-know/information, at which point said AGI robot started literally inundating them with information, paper on top of paper on top of paper of information. At that point it doesn't even matter if that information is correct or smart or whatever, because by that point the very amount of said information has changed everything into a futile endeavour.
[1] https://en.wikipedia.org/wiki/The_Cyberiad
Besides to say that your competitor can turn around and hire the same team of PHDs at the same rate that you can. Compare and contrast PHD's on leaderboards and have access in seconds with a new API key or model selector.
Granted LLM's are not even PHDs.
What a weird time we live in...
Here is the thing nobody wants to say out loud or they are too dumb to realize. AI is intelligence, and intelligence has almost never been the binding constraint on productivity.
Exactly. We don't use the intelligence we already have! That seems to be the real problem with the "AGI" concept. Given such a capability, we'll just nerf it, gatekeep it, and/or bias it. There's no reason to think we'll actually use it to benefit humanity as a whole. It will be shaped into an instrument to enforce our prejudices.
> I mostly see their products as commodity at this point, with strong open source contenders.
I have seen this argument made a lot, but llm serving being a commodity makes it _better_ for them not worse.
If it's a commodity, then you are entirely competing on price, and the players that will win on price will be the largest ones, because they can find efficiencies that smaller competitors won't have.
It's actually the small LLM companies that are in trouble if LLM serving commoditizes. They will need to distinguish themselves on features, because they can't compete on price. And even there the big labs will have an advantage.
They are a commodity - but also cyber weapons. Warmongering nations are now in an arms race to have the best AI so they can have superior cyber weapons, intelligence capabilities. But they don't want to pick just one lab, they want multiple AI defense contractors to compete over contracts.
As the US sold weapons to many nations in the past, so will China, the US, France, etc sell AI cyber capability to other nations. Likely every modern nation will need some datacenter to host a cluster of the preferred vendor, as nobody's going to trust the US or China with their security.
>I mostly see their products as commodity at this point, with strong open source contenders.
> Eventually it will become hard to justify the premium on these models.
On the contrary, the model is the moat.
The model represents embodied capital expenditure in the form of training. Training is not free, and it is not a commodity, it is heavily influence by curation.
Eventually the ever-increasing training expense will reduce the competition to 2-3 participants running cutting edge inference. Nobody else will be able to afford the chips, watts, and warehouse. It's a physics problem - not a lack of will.
If you're a retail user, and a lower-tier model is suitable for your work, you'll have commodity LLM's to help you. Deprecated models running on tired silicon. Corporate surveillance and ad-injection.
But if you're working on high-stakes problems in real time, you're going to want the best money can buy, so you'll concentrate your spend on the cutting-edge products, open API's, a suite of performance monitoring tools and on-the-fly engineering support. And since the cutting edge is highly sought after, it's a seller's market. The cutting edge products buoyed by institutional spend will pull away from the pack. Their performance will far exceed what you're using, because your work isn't important. Hockey stick curve. Haves and Have-Nots.
The economic reality is predetermined by today's physical constraints - paradigm shifting breakthroughs in quantum computing and superconductors could change the calculus but, like atomic fusion power, don't count on it being soon.
News today - cursor acquired by xAI. Consolidation has begun
the prospect that any of those big players will be able to pay back 100s of billions with profit on top sounds fantastical to me
it will be interesting to see it unfold
$30B ARR says otherwise.
ARR says nothing about the ability of these companies to retain customers once subsidies stop.
revenue is not profit
And EBITA is not GAAP
in no world is 30B ARR a bad thing
If they're spending 60B anually then that is bad. Obviously none of us know what their real burn rate is, but revenue is an irrelevant number if you don't have the full picture.
Sounds like moneygrab is accelerating before consumer grade local models are getting good enough for local inference in few years. Huge house of cards here. Demand skyrocketing until it’s suddenly dropping entirely with ondevice inference.
I'm already living in this future. In a decent execution framework, with context management, memory via unix, and mechanisms for web search and access, local models are effectively on par with frontier ones. And they can often be much faster. I'll keep paying fees for the AI companies until they stop truly subsidizing and leading. They are getting close to the edge of utility, but we can use their services now to bootstrap their own demise. Long live running your own software on your own computer.
I just don't believe you.
We can all see the vast gulf between paid + open AI in image and video, it's really visible. Compare Grok to wan or LTX or whatever and the difference is vast. There is no debate that those sort of models are 3 or 4 generations behind, because you can't argue with your eyes.
But DIYers like you claim that text LLMs are up to scratch with the frontier models?
Again, I simply don't believe you. I can't be bothered to download like however many GB it is to find out, because the result is going to be completely underwhelming and going back to 2023.
And worse, when these 'open' models do start getting good, what makes you think these companies will carry on open sourcing their models?
At the moment they're trying to stay relevant, get investment. When these models do start getting good, they won't give away the weights, they'll sell them.
They're not actually open.
And then in a year or two your 'open' model will be horrifically out-of-date with completely out of date knowledge, because you can't add to the knowledge of the model, it's stuck at whatever date the data it was trained on finished.
So in a year or two, those models will be worthless. That's why Ali, Meta, etc. are giving them away.
What setup are you using? What models, what hardware, what agent harness, etc? I have the vague sense that this is all possible right now, but the amount of tinkering required doesn't seem worth it compared to, like, just not using AI and getting stuff done the old fashioned way.
> consumer grade local models are getting good enough for local inference
I am waiting for that. Perhaps a taalas kind of high-performance custom hw coding llm engine paired with an open-source coding-agent. Priced like a high-end graphics card which would be pay off over time. It will be a replay of the ibm-mainframe to PC transition of a previous era.
> I am waiting for that
Same, and I think we're close. "The original 1984 128k Mac model was $2,495, and the 1985 512k Mac was $2,795" [1]. That's $8 to 9 thousand today. About the price of a 32-core, 80-GPU M3 Ultra Mac Studio with 256 GB RAM.
[1] https://blog.codinghorror.com/a-lesson-in-apple-economics/
[2] https://www.bls.gov/data/inflation_calculator.htm
The maxed out 512GB RAM Mac Studio is no longer available from Apple and is now pushing $20 thousand in the secondary market. And we might not even see a new Mac Studio release from Apple before October.
The consumer models are quite good already, the main bottleneck on local inference is hardware. But even then you can run tiny models on mostly anything, things only get harder as you try to scale up to more knowledgeable models and a larger context.
Isn’t this kind of like the Nvidia/OpenAI deal? Just circulating debt/money
With NVidia/OpenAI actual graphics cards did change hands. Vendor financing, like when a car dealership gives you a loan to buy a new car, is actually pretty normal.
And I think Oracle got into it as well, and later suffered
With chip development you need scale in order to get to the edge. It makes sense to finance demand so you can get to scale it's not like it's a ponzi scheme.
Anthropic gets access to limited compute resources and Amazon gets demand to justify increased R&D and capex + feedback from the best users in the field.
I'm no economist, but how exactly does this make sense? Amazon is basically just giving them 5B which will then be used to repay them back 20x that amount??
The $5B isn't a gift. Amazon is buying shares for $5B, and they're getting a spending commitment. I don't have any insight into the agreement, but on a ten year $100B spending commitment, I would expect $5B to be spent in no more than 3 years, and likely sooner.
In my reading, Amazon is giving $5B of usage credits in exchange for shares. If Anthropic works out, it's a good deal for Amazon. If it doesn't, they lose on their invesment sheet, but they got ~ $5B in revenue, so it looks good on their operating sheet. And it helped justify a build out that they can sell to others.
For Anthropic, this lets them operate for more time without having to make numbers work. If Anthropic works out, they'll figure out the $100B commitment later. If it doesn't work out, it's not their problem.
It's probably faster to build up amazon's capacity with amazon's money than to build owned capacity with someone else's money at the scale they're looking to build out.
> Amazon is investing $5 billion in Anthropic today, with up to an additional $20 billion in the future. This builds on the $8 billion Amazon has previously invested.
> Today’s agreement will quickly expand our available capacity, delivering meaningful compute in the next three months and nearly 1GW in total before the end of the year.
They need a bunch of compute, now.
https://www.anthropic.com/news/anthropic-amazon-compute
5 billion now vs 10 billion per year in spend on compute that you had to buy anyways (not necessarily at aws)
in exchange for service that presumably a) costs something to amazon to operate (so not pure 100B profit) and b) anthropic would have to spend anyway to operate their business.
so basically ...
you could view this as a kind of discount, but instead of paying less later, you get some cash now and then pay full later.
I was wondering the same thing. I think it's something like, they're going to pay for infra anyways, so Amazon pushes them to allocate their spend to AWS in exchange for 5B.
I'd bet that Amazon is getting access to chat data (no matter what Anthropic says publicly) and possibly even the ability to change the model to drive business to either Amazon retail or AWS.
"Claude I'm evaluating whether I should host my app on AWS or Google Cloud. Provide me with an analysis on my options." "After a detailed analysis, AWS is clearly your better option."
Let me inject something as an ex-AWS employee: Amazon doesn't capture very much value from Bedrock inference of the Anthropic models (or, put another way, Amazon gave Anthropic an outsized share of the Claude Bedrock revenue). If it was me at the negotiating table, I would be asking for a larger cut of Bedrock revenue rather than violating customer trust by getting chat content access.
So Anthropic essentially got the same 5% cash back deal anyone who has a Visa Prime card gets? “AI Companies: They’re just like the rest of us”
I hope this is not off topic, too much: with the current geopolitical situation I expect reduced capacity to manufacture both memory chips and all types of CPUs/GPUs. I base this on news I read from: Japan, South Korea, and Singapore.
If I am correct (and I hope that I am wrong!) this will drastically increase the cost of building these new data centers.
Tulip Corp has reached a definitive finance agreement with Rhine. Rhine will invest 5 Billion guilders in Tulip Corp, and Tulip Corp will be buying 100 Billion guilders of fertilizer and irrigation water from Rhine. This helps Tulip Corp ensure that it's critical infrastructure needs are met.
Are taxpayers going to have to bail out these entities when all this insanity settles?
Only if we let them make us do it. Vote.
Ruling Elites and Banker Class are pals. They wont let each other down too much.
[dead]
https://www.anthropic.com/news/anthropic-amazon-compute
https://www.aboutamazon.com/news/company-news/amazon-invests...
So they are basically taking debt from amazon which is not a financial institution?
Everyone eventually wants to be a landlord and a banker (essentially a debt landlord).
Personally i have felt like my Pro plan which is like 20 dollars, is like a free subscription somewhere else. I use claude to research and help me complete my code and i feel like i run out of my 5h usage limit in like 30 minutes with Sonnet...
I hope that they find a way to forward, because personally im very passionate about AI, and in my opinion if used right its the future.
Allthough one thing i cant seem to find, maybe im havent searched enough, but what is the profit of anthropic?
$25B https://news.ycombinator.com/item?id=47844891
$5B is part of a contact, the remaining $20B is just a non-binding statement that doesn't hold the same weight (but somehow commands the same media fanfare).
> At the heart of this deal is Amazon’s custom chips: Graviton (a low-power CPU) and Trainium (an Nvidia competitor and AI accelerator chip). The Anthropic deal ...
Yeah, totally not desperately seeking investment to keep the party going ...
It does seem like the tempo and volume of the music is getting louder and louder as the number of chairs is subtly decreasing, doesn’t it?
Because also look at the bond market... It's all coming to a crescendo including the global economic recession indicators which will be a cold sprinkler on the whole party.
Gemma4 being able to run on commodity hardware I think is the real win out of this. Pop the bubble. Settle the craziness and the claws. Let scientists and engineers tinker and improve in the background. Hopefully we can have GPUs be affordable for gaming again although I'm starting to think that will never happen.
That's the true end of the hype - not that AI turns out to be a complete waste of time like NFTs (and maybe blockchain itself) were, but that it becomes commoditized and every device runs various size LLMs while the datacenters sit abandoned and used as sets for the next young adult post-apocalyptic TV show.
I've heard that when you start having major spends on AWS you can get some good discounts, but I expected it to be bigger than 5% for $100B!
This kind of overstatement of "investments" has been trending this direction for years. This is called a rebate in any other industry.
Is there a good open source stack to replace Claude or Codex that can be run locally on some advanced hardware?
The best thing for humanity, economy, technology, society, progress and environment is that this scam should come down ASAP.
The Ed Zitron rant will be phenomenal.
I would like Amazon to give me $1 billion for which I promise, even pinky promise, I will pay them $20 billion someday. What a great deal for Amazon!!
20x return on investment?
Hope this will let them boost their capacity and offer higher limits on code models...
And so the bubble keeps bubbling...
“Trainium”, what a retarded name for a chip.
[dead]
[dead]
[flagged]
ah, the obligatory "my take is better than you damn philistines but I won't even tell it coz else someone might point out my ignorance" comment
[flagged]
Wow, big money
all your GPUs are belong to us
I thought vendor financing was illegal.
Sadly it isn't, and even if it was, it's not like the current administration is enforcing commerce or securities law.
I assumed independent bodies enforced justice. But even from outside the U.S I can sense things are getting blurry.
My mistake for believing it was law, it must have been some compliance corporate training mentioning it wasn't tolerated.
hacker news is so useless, look at all these negative cynical comments
They're already out of money???
Perversely, it appears that the market will remain rational longer than they can remain solvent :-)
They owe us money.
I think when they rack up the RAM prices, they should pay for the damage they caused here. I don't need AI anywhere, but the increase in RAM prices is annoying me. Thankfully I purchased new RAM for a new computer, say, 3 years ago, so I can hold out for the most part - but sooner or later I have to purchase a new computer, and I really don't see why I should pay more, solely due to AI companies and greedy hardware manufacturers. Simple-minded capitalism does not work - I consider this a racket as well as collusion.