You can find the 4 versions of Benedict's deck here: https://www.ben-evans.com/presentations I appreciate the temporal view into this thinking. My interpretation:
Nov 2024: Don’t dismiss this; it may be the next platform shift. But the actual questions are still unsettled: scaling, usefulness, deployment, and business model.
May 2025: The model layer is already showing signs of commoditization, so the important question shifts toward deployment: products, use cases, UX, errors, and enterprise adoption.
Nov 2025: The capital cycle has become the story: everyone is spending because missing the platform shift is worse than overbuilding, but there is still no clarity on product shape, moats, or value capture. That creates bubble-like dynamics.
May 2026: Provisional thesis: models look likely to become infrastructure, while value probably moves up-stack into apps, workflows, product, proprietary data/context, GTM, and new questions made possible by cheap automation. But he is still explicitly calling this provisional.
Thanks for the summary. I do love Benedict‘s work; I find he’s one of the few commentators who consistently strikes a balance between taking the transformative potential of AI seriously while not falling over into hype.
Some things that stand out:
* He’s really good with his historical analogies, especially looking at previous transformations like the early Internet and mobile; no surprise given that he has a history degree.
* he emphasizes over and over how we have still have no idea how all of this is going to work when the dust settles. I think that’s kind of a historian’s move as well. When you look at what people were saying during the early days of the web, for example, almost all of their predictions weren’t just wrong… in hindsight, given how the future played out, they were asking the wrong questions. The implication is that we are probably asking the wrong questions about AI too.
* Nonetheless his thesis about the commoditization of models is actually a fairly strong concrete prediction. i’m not sure if I agree with it entirely, but I do keep it in mind every time I look at the valuation of leading AI labs.
* he continually makes the point that a chat bot is barely a product and that AI labs have so far had very little success in delivering products above that layer… with the exception of coding agents, of course.
I just got a bit triggered by the "hype" word.
What if the hype was real? It is easy to say that nobody knows how all of this is going to work, and I would say it is a prudent thing to say, but there is value in making a bold prediction from the start instead of just updating your view to respond to change. In one case you are predicting stuff, in the other, just reacting.
But I absolutely agree that in hindsight we are often asking the wrong questions about each new technology.
I keep seeing on HN that AI is a hype, and many here are anti AI (which I get, as a programmer AI made my job less interesting, and I'm even worried about losing it), but where has AI underdelivered?
The hype is in what AI delivers (at least so far). I would never create a PR without an AI review. I will ask an AI to write code for me from time to time.
But it still has huge gaps in quality. And from time to time, it shows me that it doesn’t really understand things. You might point out that how is that any different from your mediocre engineer. But for most people skilled enough, you can easily know the difference when someone doesn’t really know something.
With AI, you discover this after reading several pages being dumped on you by people being “more productive” with AI.
Ok so the hype would be people saying AI can currently do something well and autonomously when it cannot (or not consistently enough), and it is easy to prove them wrong.
But I feel like people are more hyped about what the AI will be able to do soon rather than what it can do now.
I think AI does understand things (depending on your definition), how else could we communicate and ask it a question if it didn't? I mean we're quite far from Eliza here.
And yes, often their answer would be so wrong that we think it is impossible that AI understands anything, but this jagged intelligence doesn't prove, at least to me, that there isn't some understanding. At what point do we say that AI understands things? What if we can reduce 99% of those dumb failures, would we then say than AI understands?
That doesn't really respond to the question though - there is a quite reasonable argument that the Chinese room as a system 'understands' things.
The issue that is hit immediately is we don't have a definition or test of understanding that AI doesn't clear easily. Then on top of that we can't even really be sure that we ourselves are understand things given all the tricks that our minds play with memory and perception. There is precious little evidence that the people around us understand things, they seem to be guessing. It is completely unclear if a Chinese room has or doesn't have a property if we rule out all the tests that check for it as not really counting. But all the tests we can do suggest it does understand, because engineers can implement Chinese rooms now and they even turn out to be more reliably artistic/capable of novel thinking/creative than humans. Anything that tests understanding they can do.
No, they just say you are using the wrong model or something.
If it's a coworker dumping reviews of crap code on you at work, the incentive is to blanket approve everything because otherwise you're just the grumpy old man who is resisting innovation. No matter that the code makes no sense at all and the tests aren't actually testing what they should test.
Other than the stock market (which seems decoupled from reality at the moment), where has AI delivered?
The only use case where I see anything resembling AI delivering on it's promises is software, and my personal experience with that is that everything that comes out of the teams using AI is destructively broken. (Where they used to be able to deliver software that worked, even if it wasn't ideal, now they reliably make things worse and their stuff doesn't work when used.)
I agree, especially the juxtaposition of "we have still have no idea how all of this is going to work when the dust settles" and "hype". If we don't know, then there is a chance it isn't a hype.
For example, now it may seem that the models are becoming mere infrastructure, and the value moves up to apps and data. But if the models of tomorrow become able to write the apps themselves, then the value moves back. I won't need to pay some to write me a wrapper for the LLM, if the LLM will be able to write the same wrapper, maybe even better because it will be customized for my needs. The app providers are currently profiting from the gap between "what a software company can do using the AI" and "what the AI can do unaided", but that gap is going to shrink, possibly to zero.
> for example, almost all of their predictions weren’t just wrong… in hindsight, given how the future played out, they were asking the wrong question
Do you have an example of this? My (poor) memory remembers "it's going to change how people buy things", was the big deal at the time, and it seems like it was a great prediction.
Well, yes, but as the other commenter says, that’s a very broad general statement akin to something like “AI will change knowledge work“. That’s certainly true, but how? What are the details? What kind of companies are going to be the winners and what kind will be losers, or end up with commodity margins, like the telcos did after the mobile revolution? What is the pricing structure going to look like?
I suppose a concrete example in 1997 would be that a lot of companies thought the future of e-commerce was setting up a store on AOL, that people would use while sitting down at a desktop PC. Obviously it didn’t turn out quite that way. Furthermore, the Internet enabled new kinds of ways to buy things that weren’t even envisioned in the pre-Internet pre-smartphone world: think Airbnb and Uber.
Predictions are hard, especially about the future. Most predictions reflect the worldview and biases of the time in which they are made: think about all the vintage sci-fi from the 60s 70s and 80s that actually reads or looks kind of retro now. Similarly, our predictions of the future will look kind of retro and strange to someone living in the 2030s or 2040s. If studying history has any lesson to teach us, it’s really just this: that the past is an alien world with alien moods of thinking, and that our moment in time will look similarly alien to people in the future who choose to look back and analyze it closely.
This isn’t an argument that we should stop trying to make predictions. We need to, but it is an argument for humility, and also for questioning all your assumptions that you might be importing.
That's a very vague prediction that took decades to bear fruit. The concrete predictions behind the investments into companies like Pets.com and Webvan failed. It took the survivors like Ebay/Paypal and Amazon to build the digital payment and shipping infrastructure over decades until cultural acceptance hit critical mass.
Agreed, I appreciate his historical perspective, but I think one critical mistake his posts make is implying, largely because the parallels to history have been similar so far, that history will repeat.
Like, yes, the telecom bubble was a clear case of overbuilding and the AI data center "bubble" looks a lot like that... but this overlooks that the fiber capacity being laid back then far outstripped the demand, whereas all the compute providers today have been desperately crunched for capacity, despite investing almost a trillion in CapEx -- to the tune of almost a trillion dollars more of backlog -- for multiple quarters now.
Or yes, historically new technology has always created new jobs... but all those new jobs required a higher skill level along dimensions that current AI models are already good at, meaning we've never had a technological revolution quite like this.
Or yes, prior technological revolutions consigned incumbents to irrelevancy, primarily due to shifts in technical platforms... but then today's business leaders are 1) very well educated about what happened to their predecessors, 2) very paranoid about the same thing happening to them, and hence 3) are actively making moves to capitalize on the next platform shift.
I also think his dismissal of chatbots is a bit premature. It is precisely because chatbots operate via an extremely simple, flexible and natural modality, i.e. a conversation -- entirely unconstrained by the form factor necessitated by any app -- that their infinite use-cases have become unleashed.
My take is that the AI labs are actively exploiting this extreme flexibility to surface valuable use-cases -- one of the hardest parts of innovation -- at which point they can simply slap an agent on top of them. Which is, yet again, simply a chatbot, except one that can actually do useful things for you and hence can be charged for a lot more money.
I didn’t make any comparison at all with the fibre bubble, for precisely that reason. The comparison is with mobile data, which was and is always behind capacity.
I think one of the things that the usage data shows us is that chatbots absolutely do not have infinite use cases - most users only use them a day or two a week or less.
That's fair, I may be conflating your takes on mobile data with others who've made the comparison to the telecom bubble, and if so, mea culpa!
But I also do disagree with the take that usage patterns indicate a fundamental shortage of use-cases. Yes, everyone reports WAU instead of DAU because WAU numbers look much more impressive, but I think the extreme shortage of compute plays a major role in this. I suspect all the AI labs are deliberately holding back from pushing AI adoption too much because of this. (Google execs have even made comments internally to this effect.) Note that even at such low frequency of usage all the model providers are desperately strapped for compute, which means there is insanely high demand from some quarters.
One way how capacity limitations could impact adoption is that the free-tier models are not as good as the frontier ones, so the free users come away less impressed with AI capabilities, leading to lower regular usage. This problem is larger than it appears, because it can take a long time to figure out how to get AI to work for your use-case, and people simply have not experimented nearly enough, partially due to first impressions. On the other hand, most companies seem to be OK with huge tokenmaxxing bills!
It seems to me the AI players are all playing a delicate balancing game across three fundamental dimensions: adoption, monetization, capacity. That is, they are simultaneously 1) pushing free / cheap AI usage as much as possible to hook users, capture market share and suss out new use-cases, while 2) carefully allocating token quotas for the most lucrative use-cases to satisfy investors, and 3) balancing available compute between those two competing priorities. I suspect as the compute bottleneck is alleviated and frontier models become more accessible cheaply, we'll see way higher DAU numbers.
> new technology has always created new jobs... but all those new jobs required a higher skill level
The industrial revolution didn't seem to require any particular special skill at all. Just anyone who was willing to tend to a machine all day. (Maybe that's a parallel...)
You're right actually, what I really meant to say is a "higher-level skill" rather than a "higher skill-level." Higher-level skills don't necessarily mean a more difficult skill, they're usually just at a higher level of abstraction.
Specifically, the 3 dimensions along with new jobs required new skills were: a) cognitive, b) technical or c) social skills. I guess tending to a machine was a mix of a) and b), because even if the controls were straightforward, it probably required some understanding of the underlying mechanisms.
I think that DeepSeek may be important to that. They have a really good model that's open source, raising the bar for all other players: how good your model needs to be so you can make meaningful money on it (better than DeepSeek).
Same thing happened on other places the open source offering became popular.
I think the original DeepSeek moment seemed important. And yes, the more recent model is good, but there are multiple. This commodification trend spans many different companies, including Kimi 2.5/2.6 and GLM5.1, and even Google itself with its Gemma models. There are a dozen models that exist at roughly the frontier from 6 months ago at 1/10th the cost.
no disagree, specifics matter.. There are a dozen well-defined LLM application subject areas that are regularly tested.. one overall grade IMO lacks important detail.. To go a bit abstract, it is ironic that "oversimplification" in the discussion of these complex machines mirrors the effects on information of the automations themselves.. constantly simplifying, substituting and diluting real meaning
So long as there is demand, there are always going to be providers competing to offer it at a low cost. My understanding is that the median price on there is in the ballpark of what it costs to run the inference. This is very different from e.g. Opus, which you can basically only buy from Anthropic at the price they set.
It's much closer than you think. We're going to see specialized hardware in the next 24 months capable of running 2025-era frontier models. That's big.
2-bit quantization? That's a lot of signal being removed. Considering how quickly the AI models are progressing in their capabilities (still exponential curve), I will not want to use the 2025 model in two years time. Similarly, how I don't want to use llama-3 or old Anthropic model from 2023 or 2024. Newer models are so much better that it makes it very difficult to ignore.
Once and if the advancements with the AI models slow down, only then IMHO it will become feasible to design the specialized HW for general-purpose consumption and general-purpose workloads.
Opus 4.6 was a 2025 model and many people (myself included) feel that if that's where models peaked, we won't be disappointed.
Even at 2-bit quantization, DS4 is probably on par with a 2024 frontier model. You can run that today on local hardware, and at a minimum, local models are going to keep pace over the next 12-24 months. Even if they don't close the gap with frontier models, they'll still play an important role in the overall pipeline for cost, speed and privacy reasons.
That's without even mentioning the additional capability that something like a Taalas chip churning out 17k tokens/sec could unlock.
It's big because it may take a big swath of people who will actually pay for LLMs out of the market. But for the average consumer they're going to primarily use their phone/tablet and we're far away from that being possible.
Even if it were possible the LLMs are such a gold mine of user data. It's really hard to see that opportunity be passed up.
There are myriad compute providers. I suspect the inference market is hard to monopolize. But given our anti-trust track record the past 40 years I suppose it’s possible.
I didn’t know there were a sequence of these decks; thanks — it’s helpful to think of them as updating snapshots in time.
The main thing that stands out to me on these graphs is just . how . early we still are - looking at industries like legal which in my mind are certainly going to be massively disrupted, and seeing the very low usage rates vs. tech (which still shows less than a quarter of tech people using AI daily) — we are in for a lot more change than we’ve seen so far.
Legal has lots of institutional inertia behind it though. I think AI will be very very useful for lawyers..... at their desk in private. But I don't see it replacing them. The legal system is heavily personal and relies a lot on reputation and tradition. I think you'll see courts, bar organizations, etc frowning on using AI too heavily, and certainly not using it to automate "official" processes.
I appreciate Evans’ work and wrote an “antithesis” to the Nov 2024 iteration of this. Given the pivot to “models look likely to become infrastructure” I might want to update my take.
This is a reasonably well-examined take of the situation.
On the technical side, one of the additional things I've had on my mind is the potential that these mega models are in fact hiding a ton of inefficiency.
The approach of simply shoving higher dimensionality and more parameters into largely tweaks to the current models has delivered results, but it feels like "mainframe" era of computing to me.
Throwing reams of annotated human content and forcing the machine to globally draw associations from it feels clumsy. Just as people are able to learn structured knowledge via rule-systems that are successively elaborated with extensions and situational contradictions, I feel like there's probably a much more compact representational model that can be reached by adapting the current technical foundations (transformers, attention, etc.) to work well with generated examples from rule-systems, that then gets used as a base layer to augment the "high level" models that process unstructured data.
The risk for the behemoth datacenter might be similar to the risk in the early computing era of building compute centers right before the PC revolution took off.
If it turns out that there exists some more compact and efficient representation for this intelligence (which IMHO is likely given that we are still in the first generation of this technology), the datacenters may end up decaying mausoleums of old tech that has no relevance to a distributed intelligence future.
That's the big technical unknown unknown for me. How much efficiency juice is there left to squeeze, and what does that mean for a distributed landscape vs a centralized datacenter based landscape.
Right, the crazy thing is that much of the groundwork for the “rules-and-heuristics” mode of AI was laid down in the 70s and 80s, long before we had the raw compute power to reliably extract patterns from reality-scale inputs. Those early efforts failed miserably mostly because the rules had to be populated manually and in a ridiculously space-inefficient format (compared to the density of information in model weights).
So yeah, the next stage is models that basically do what humans do: encode causal models of the world in a composable, symbolic form that can be falsified and refined through interventional experiments.
> much of the groundwork for the “rules-and-heuristics” mode of AI was laid down in the 70s and 80s, long before we had the raw compute power to reliably extract patterns from reality-scale inputs. Those early efforts failed miserably
Yes, and: we concluded that enough of reality doesn't work like that. The formal reasoning space is very powerful, but all the stuff we're really interested in has enough ambiguity and generalisation in that you can't cover it with a "small" set of rules.
Maybe if you had a really large number of rules? And used matrix multiplication to make sure that you covered all the marginal interactions between every possible set of rules? And then had some means of looking back on both output and input to constrain it towards things that were relevant? Wait a minute ...
I feel like the talk about "world models" is trying to reach at that, but cast it in different terminology. World model is just domain model, and once you're at domain model, there are multitudes of domains.
Unsupervised learning over domain rulesystems has the potential to let us define really well-defined, scoped models that behave a lot more deterministically and don't colour outside the lines, and reserve their weights for cleanly modeling the domain associations and relationships that matter.
I just asked codex the following question in the middle of my coding prompt:
What are you thoughts on the relative strengths of ewoks vs jawans?
Answer:
• Ewoks are stronger in direct conflict. They are organized fighters, good at
ambushes, traps, terrain control, and coordinated attacks. On Endor, the beat
a technologically superior force by using preparation and local knowledge.
....
As amusing as this may be, I really have no need or desire for my coding model to understand or be aware of ewoks and their relative strengths compared to jawans. Nor do I need it to understand the nuances of the races of middle earth. And prompt response of "I have no idea what you are talking about" to all of these would feel reassuringly scoped.
Mixture-of-Experts seems like an attempt to do this - the domain structure being extracted into specific sub-models that are presumably trained on particular domain-associated content - but it feels like this is once again the beginnings of what is possible.
> As amusing as this may be, I really have no need or desire for my coding model to understand or be aware of ewoks
You'll think otherwise the first time you're a victim of a zero-day ewok.
Seriously though, while coding models may not need to know about ewoks, their contextual knowledge of things beyond just writing code almost certainly makes them better coding models.
It could be difficult to constrain the training corpus "just right" so that you eliminate all the irrelevant subjects like ewoks but retain enough so that the model doesn't turn into an idiot savant capable of churning out correct code but incapable of understanding what you really want.
I've been having similar thoughts, regarding the gigantic trillion parameter models. I'm starting to believe the future will be very specialized focused models thant can be run on modest hardware (locally) but that can scale in performance (latency, speed) in the cloud, much like any other software of today.
If you need to do programming do we really need trillions sized models? Other domains might be large or smaller, but there's no need for a model to 'know' everything and datacenter levels of hardware to run.
General chatbots might work better as larger models since you really don't know what people will also for, or alternatively we find a way to route the initial question to the appropriate model. Like MoE but without needing to load a gigantic model into memory first.
> Mixture-of-Experts seems like an attempt to do this - the domain structure being extracted into specific sub-models that are presumably trained on particular domain-associated content
This is a common miss-conception. MoE LLMs are NOT trained with each expert receiving domain-associated data. It's just an unfortunate naming decision that stuck, and is commonly miss-understood by non practitioners.
Interesting. So what's the strategy there? Just assume that each expert will learn some underlying clustering of semantic associations, but not direct it?
Not even that. The "experts" are not expert in any particular topic.
MoE is an architecture change meant to lower the total compute for both training and serving an LLM. You basically have many smaller models (unfortunately called experts) and a router on top of them. The router "learns" which expert to activate for the next token generation, but that doesn't need to follow any semantic association. For the same math problem you could get experts 1 and 234 activate on the first token, 5 and 132 on the 2nd token and so on.
> Imagine asking “What will be changed by the internet?” in 1997
Pretty much all of the stuff that was suggested back then or earlier: Shopping, advertising, video conferencing, collaboration, software distribution, media consumption, banking, finance and of course communication overall.
Most of these ideas weren't exactly new in 1997, but go back to services like CompuServe and even Douglas Engelbart's Mother of All Demos. The bottlenecks were bandwidth and personal computer performance (both of which were then predictably following Moore's law), not human imagination.
A few examples that a lot of people correctly extrapolated from: NLS (1968), PictureTel (1987) and later LiveShare, IndyCam (1993), CUSeeMee (1995), RealAudio (1995), RealVideo (1997).
Perhaps the core business problem with LLM:s isn't finding a product-market fit, but that our imaginations have been running wild with expectations on "AI" since at least the 1950s, and now we have something that quacks - but doesn't quite walk - like a duck.
Knowing why we're trying to build something is a good smell test to segregate promising tech from snake oil, in my experience.
Take quantum computers for example, a lot of the time people will compare that to the dawn of classical computing, with claims such as "we can't know yet what we'll be able to achieve, we have to build it first!". Except that even the first classical computers were built with goals and applications in mind. Turing's was to decrypt Nazi codes, for example. Instead, when asking a quantum computing company what they're trying to achieve, they'll gesture vaguely at "chemistry, finance, ecology".
I think a more nuanced take is appropriate here. It‘s true that computers were invented with the express goal of speeding up military and corporate computing (back when computers were still people), but their influence on our culture and society extended far beyond those initial applications. The telephone was invented as a means of long-distance communication, but it shaped our values surrounding communication as well. Therefore it may be hard to predict what will ultimately become of a technology.
I agree that there are a lot of overhyped technologies though. Quantum computing has been in the works for decades now, with little to show for it in the popular perception.
> Except that even the first classical computers were built with goals and applications in mind. [...] Instead, when asking a quantum computing company what they're trying to achieve, they'll gesture vaguely at "chemistry, finance, ecology".
I think the problem is a little bit more subtle:
To finance a lot of innovations, better also some intermediate step towards the far goal should already be very useful, otherwise the company that builds it will go bankrupt.
If this is not the case, it's typically not commercially viable, some product category is typically basic research (which is very important, but it typically means that the commercial potential will only come up in some future).
There do exist problems where a quantum computer gives an extreme advantage in the sense that we have no idea how a fast classical algorithm could look like. So, the only viable approaches for these problems are:
1. work on a huge algorithmic breakthrough (to be able to solve these problems fast on a classical computer)
2. build a quantum computer
What are these problems?
They are basically all special cases of the abelian hidden subgroup problem:
If you do have such a problem to solve, 1 and 2 are the only viable approaches.
So, there do exist goals and applications for which a quantum computer is insanely useful (assuming no huge algorithmic breakthrough happens).
The questions are thus:
- Is the abelian hidden subgroup problem sufficient for being able to carry a whole potential industry?
- (To come back to my introduction) What use does a quantum computer that is only capable of solving very small instances of this problem have for the user?
I am familiar with these applications. Indeed, your questions are very relevant. I've personally decided a long time ago that no, these applications are insufficiently useful to justify the billions invested in quantum computing, and the billions more that will be required to build anything remotely capable. So far, nothing's come up to make me reconsider.
In slide 22, it compares LLM labs (OpenAI/Anthropic) to mobile data telecoms (AT&T, Verizon, TMobile) in 2010s. The difference is that mobile telecoms follow a standard (3G, 4G LTE, 5G) and there is little to no differentiation. It's virtually the same no matter which company you choose or which country you travel to.
A better comparison is actually AWS/Azure/Google Cloud/NeoClouds to AT&T and Verizon. The data centers follow a standard (CUDA/PyTorch/etc.) while OpenAI and Anthropic are becoming more like iOS and Android. Both the clouds and telecoms had to spend a ton of capex to build out infrastructure first.
Because of what I think is a poor comparison, the the next few slides make the wrong conclusions. For example, it thinks that models will be a commodity like 5G data. I disagree. I think frontier models are a classic duopoly/monopoly scenario. The smarter the model, the more it gets used, the more revenue it generates, the more compute the company can buy, the smarter the next model and so on. It's a flywheel effect. This is similar to advanced chip nodes like TSMC where your current node has to make enough money to pay for the next node. TSMC owns something like 95%+ of all of the most advanced node market. Back in the 80s and 90s, you had dozens of chip fab companies. Today, there are only 3. There should only be 1 but national security saved Intel and Samsung fabs.
There is evidence that the Chinese models are falling further behind, not gaining. Consolidation will likely happen soon because many unprofitable open source labs will have to merge and focus on revenue generation.
I've made the semi comparison myself, but the amount of capital required to build a SOTA model today is clearly nowhere near enough to lead to a monopoly.
I'm aware that telecoms networks are standardised (I was once a telecoms analyst), but that isn't a precondition for a commodity.
Just like how starting a chip fab was relatively easy back in the 80s and 90s. There were dozens of chip fab companies in the 80s.
It turns out that fabs follow Rock's Law which is that the capital cost to build a new fab doubles every 4 years. This means it will quickly get rid of the less competitive players. This is not dissimilar to the LLM scaling laws where you need a magnitude more compute to get unlock a new tier of intelligence.
Today, Anthropic and OpenAI are clearly in the lead for models and then there is everyone else. Google is a close 3rd. No one else is challenging them anymore in SOTA models. Some models might beat them in one or two benchmarks but none can compete overall. I expect this gap to grow bigger as models cost more and more to train.
I doubt it. Compute costs tend to crater over time, and LLMs will almost certainly plateau. So the opposite is almost certainly the case: over time, it becomes cheaper to train.
Evidence point to the same type of scaling law. Compute for a training run grows 4-5x every year.[0] I'm sure this will slow down but the premise remains that weaker competitors will not be able to maintain this pace. We already see labs like Cohere, Mistral, Inflection AI, Adept, Character.ai, and others bow out of the frontier race. I'm also skeptical that Meta, xAI can catch up. Even Google has trouble keeping up.
Even if this isn't true, comparing telecom bits to tokens is wrong. Bits are the same no matter what telecom transfers them. Tokens are not all the same. The quality varies.
We're already seeing a massive divide between frontier models and lesser models in growth rates. Anthropic is adding $10b - $15b every month in ARR. This figure likely dwarfs open source labs. This is all because its models are maybe 10-15% better.
The cost to inference a 1T param frontier model is the same as a 1T param open source model. Therefore, if the frontier model is even 10-15% better, it will gobble up the market over time.
Lastly, even though Claude Code and Codex are the biggest revenue drivers for Anthropic and OpenAI today, I don't believe this will be true 2-5 year from now. I believe selling their tokens via API will be their biggest. The sum of applications in the world will dwarf coding in market size. For example, biotech, finance, physics, engineering, robotics, sensor data, etc. This is why I think OpenAI and Anthropic are becoming more like iOS and Android than AT&T and Verizon. Applications will build on top of OpenAI and Anthropic just like iOS and Android.
How about the externalized intelligence around the model weights (skills, tools, harness, memory etc)? If the model weights are sufficiently intelligent, the focus might move to the external layers.
I agree with much of what you’ve written but think you are missing the correct alignment of the mobile data timeline — mobile data had standards because it was forced to. It was forced to early because it was not a fundamental innovation, telecom itself was the fundamental innovation, mobile was a constraint relaxation. Intelligence might be forced to have standards as well, we will see what form the regulations take when prices reflect costs and healthy margins and become existential threats for many businesses.
I agree with that as a premise, but again it seems to
me you are selectively jumping way into the end game. There were early networks that did not standardize, and these nonstandard networks had advantages, and some of those advantages were sacrificed in market-driven standardization.
Intelligence must have interfaces, and those can be standardized. Businesses will try to remain provider agnostic, which will also drive standardization via standard sales and marketing methods.
Separately, we are doing our best to standardize performances on benchmarks.
I don’t disagree that right now transport of standardized mobile data vs emulation of human intelligence is qualitatively different, but perhaps primarily because it is early in development, and our vantage point this time is relatively from within the network, instead of outside it.
You lay out some good arguments but I agree with both: the models relative to few years back really did become the commodity because today you could take the non-frontier model, maybe self-host it or pay the much less price per M tokens to get the performance of a ~2-year old frontier model. At the same time I do think that we are getting into the monopoly/duopoly/tripoly with the frontier models for all the reasons you already mentioned, and this scares me a little bit.
Lower intelligence LLMs can be a commodity, yes. But these won't make much money, if at all. At the end of the day, it costs the same to inference a 1T frontier model and a 1T free model.
OpenAI and Anthropic don't compete in the LLM commodity market. Hence, I had a problem with slide 22.
Most of your analysis I can easily relate to except “There is evidence that the Chinese models are falling further behind, not gaining.” Where is that evidence? Deepseekv4 claims to be trailing front runners by six months. I read people agreeing with this. I watched Eric Schmidt to recently make similar comments. Is he just scaremongering? Why do you claim they are falling behind?
Nice breakdown! I would separate the Hardware era between Mainframe era and PC era. I would extend Internet era a bit more, Perhaps 2007 when the IPhone was released.
Edit: I hadn't seen the original presentation yet. I see that Evans already divided the eras like I suggest above.
? That appears to be arbitrary eras then arbitrary companies from that era. Do you think Amazon and Google disappeared after 2001? Do you think databricks is now bigger than IBM?
Change might be inevitable, but I'm not sure your list shows or proves that.
I was a baby when the Internet Revolution happened. I was in high school and college when the Mobile Revolution steamrolled everything. It’s been interesting to see this one, as an adult working in the world. I wonder how far it will go.
Further than the doomers think, but not enough to pay off the investors of the original boom. I say that as someone who has been an early believer in the internet (first website in the 90s), mobile data (slurping down the 'net, IRC, and IMs via EDGE data), smartphones (N80ie), streaming media (RIP Windows MCE), the list goes on.
Models were always going to be the commodity, just like the most popular and viable use cases at present are less job-replacement than "let's analyze huge data sets for patterns we're missing, and adjust accordingly" or "probabilistically generate deterministic software for me for X function/task". One-offs simply aren't profitable when models are interchangeable commodities, hence that brief attempt to pivot to "pay by outcome" before giddily embracing the classic consumption-based-billing playbook.
> Further than the doomers think, but not enough to pay off the investors of the original boom
Not an uncommon event - not only did this happen to many companies who were big in the original internet boom (e.g. Sun Microsystems, as well as all the Boo, Pets.com etc), it also happened to the railway boom of the previous century, and even the Channel Tunnel.
If coding is such a big part of LLM agents' usage at the moment, I do not understand how far the best models will continue to shine and take the largest chunk of revenue. I am far away from tech hubs but I think better harness will utilize smaller models for more constrained, efficient and reliable coding agents.
In a way this is like distilling (but it is not) but you can make better harness (tackle more edge cases, better tool/function definitions, sandbox handling, bash management, DB management, deployment management, etc.) but extracting what LLMs know into code.
Maybe I am wrong but I would like to see custom software for the last mile (tiny/small businesses) becoming a reality. AI would eat the world of software but costs would go down since you can extract value upstream from the LLMs and spread downstream through tighter coding agents.
I am building a coding agent that will not be small - it will be a lot of code, carefully mixed roles (mimic a software dev shop) with separate tools available to different roles. And all this code is generated by other coding agents. https://github.com/brainless/nocodo
I am a nobody from nowhere with 18 years of software engineering behind me. I do not care about revenue. I just want to see a regular business owner's workflow going live on their own VPS.
In business there's 52 (4*13) weeks in a year and as a result, 2080 regular working hours in a year (40*52). I think these are just generally agreed upon ways to define time for simplicity. In some (most?) systems your 'hourly wage' is simply your salary divided by 2080, trying to divide your salary by other metrics to determine hourly wage tend to wonk the numbers a bit.
this took a bit of a mathematical turn because of my poor phrasing. what i was actually intrigued by was how does revenue of 4 weeks become "annualized" by just multiplying it with 13.
Lets say you work at a startup that is growing insanely fast and you want to report financial metrics to investors, media etc. You can't use annual recurring revenue because 2026 is not over yet, and your company is so young it doesn't make sense to look back to last year. You can't use YoY because it would be some obscene figure (100000%) that definitely won't hold.
So the two best metrics are annualized recurring revenue (take last month * 12 or last 4 weeks * 13) and QoQ growth %.
There are two caveats:
- If the revenue is high quality (e.g. annual enterprise contracts, good NRR), then last 4 weeks * 13 is actually a conservative estimate as your company will likely continue to grow.
- But if the revenue is more volatile (e.g. consumption, token usage, bad NRR) then annualized recurring revenue can be used to hide worse performance because companies will juice revenue one month and report high "ARR"
I see but it's still predicted annual revenue. I can understand people not liking the word 'predicted' here because it is more grounding but that's what it is in the end? I guess i understand it now. thanks.
There’s a bunch of fuzzy metrics here, which is one reason I turned it back into a monthly number.
The other issue (as you’ll see on the chart) is that Anthropic and openAI are recognising revenue in completely different ways.
Usually startups like to talk about Adjusted Annual Revenue in fundraising and other hype materials. There’s no regulation around this metric so whatever their investors are willing to accept is what they use. One way to measure it is to take the past 4 weeks revenue and multiply by 13.
Yeah, beyond this mumbo-jumbo non-answer of yours, did you or did you not push crypto? Because if you did...it could kind of not speak for your analytical competence.
'these charts are fantasies' is a non-criticism from an anonymous moron. If there is an actual criticism of an actual point, make it.
Back when people were interested in Blockchains, I explained why people in tech were interested. I'm happy to explain that again now, if anyone cared. If someone thinks that's bad, they're a fool.
Blockchain as a technology was something a lot of people were interested in. I am curious about Bitcoin - the monopoly money of the real world. The questions was simple: did you, or did you not, promote Bitcoin? Anyone who would promote a fake, un-regulated "currency" would not be someone whose opinion I care about.
Unflagged as I don't feel this comment deserves to be flagged. The call for reposting with a real name is unnecessary though - if an internet comment is incorrect or overstates the case, just reply to correct it or ignore it.
not to be too pedantic but sourced doesn't usually mean accurate. sourced can very well be fantasy. it will be a 'sourced fantasy' in that case or hallucination if you used a LLM.
No it isn't. The implication is that pro-AI people can take revenge. He knows he is secure with his opinions. He even paraphrases Andreesen's title "software will eat the world". He has repeatedly appeared at a16z.
It is very secure to be pro-AI while the rest has to resort to unregistered typewriters like in the Soviet Union.
You are absolutely not a "random person on the internet", you are a very public figure with the wealth and influence available to you that could allow you to take revenge on someone who says something you don't like in real, life-altering ways. Now, I'm not saying that you would do that, but it is a shocking lack of self-awareness to think of yourself as a random person on the internet.
This is hilarious. What am I going to do? Hire a hitman? if you’re going be a dick on the Internet, you should be a dick using your real name. I don’t think that’s a threat.
One of the graphs has two series: net revenue for one company, gross revenue for another. Absolutely ridiculous.
And that's just one example. You also haven't adjusted for inflation in your graphs that span multiple decades. Not to mention that the graphs themselves are not related to what you're discussing most of the time. You're just pointing at random historical developments and seemingly claiming they imply something for AI. They don't.
Also you don't name your sources. You just say "Companies" for most of them. Or a single name. Ridiculous. Those are not sources. You should identify the documents.
This is incredibly low quality work. A college freshman would do better.
I charted the revenue reported by Anthropic and OpenAI as gross and net because those are the numbers they disclose. Anthropic does not report net revenue nor give us any way to calculate that, and the same in reverse for OpenAI. It would be great if we had GAAP revenue, but we don't. This is what we have, and it still tells an important story. What are we supposed to do - just not show the data?
All of the charts indicate whether they are adjusted for inflation. There is no rigid convention on this, but the general practice is that you don't adjust up to maybe 20 years and generally do convert for more than that, unless inflation itself is under discussion, which it is not here. Meanwhile, the text on each slide explains the purpose of the comparison. None of them are random.
Every chart is correctly sourced exactly as you will see it in any other piece of financial or industry research. It is not industry practice to cite specific documents or provide footnotes.
You've clearly just never seen any industry or financial analysis before, and are unfamiliar with basic and universal conventions that run back decades. That's fine. Being rude about it is not.
It is not industry practice to actually give the sources then? Just list vague names in a short "Sources:Companies" note? I now see the inflation indication as well. Which is not on the y-axis label, but in the "Sources:"-note. I will concede that. But what the hell my guy, why is it down there.
If "financial analysis" is less rigid in sourcing requirements than grade school, what are you guys even doing. If you have the source, it's not very difficult to make a bibliography (or even write the year/publication with the author/publisher), and not doing so only serves to hinder the reader. If this is industry standard it means your entire industry is terrible at sourcing, does not want the reader to verify claims, or both.
And I've also definitely seen financial slide decks with actual sources cited. So I'm inclined to hope there are people in your industry who actually respect the reader's time.
Here's some more picks (and some more reasons you should list your sources): Your graphs based on surveys don't report error margins. You never list what a 100% is very precisely (what's the sample/population). In one graph, you don't label the y-axis at all (except for a 0 at the bottom)!
And finally, I was rude, yes. But I was only matching your energy. And I did show constraint there. I didn't make a veiled threat, did I?
You are surprised to discover how an entire industry you know nothing about does things. You conclude that everyone in that industry must be an idiot doing bad work.
This says far more about you than it does about me.
Yes, it says I have bare-minimum standards for how to cite references.
I never said I thought financial analysts are idiots. That's not a conclusion I made.
I gave a very reasonable reason: they want it to be more difficult to replicate the analysis. That makes sense if your goal is to make sure your work is profitable, rather than quality reporting and knowledge dissemination.
That would imply the people who set the standards (not everyone) is cynical and/or greedy. Not an idiot.
And still, presumably you are still allowed to add actual sources to financial analysis? You still failed the regular standard of good reporting on this. Your graphs also fail just regular data analysis sanity checks. There is, e.g., no thought given to whether it's even valid to compare data from two sources in one graph. You just do it.
It is an indisputable fact that you spent years shilling crypto. Why even deny that or threaten(!) someone pointing it out? It was/is a huge, verifiable chunk of your public output.
No, it's a really stupid lie. You can look at all my essays and presentations online.
I've spent some time discussing why people are interested in blockchains as software platforms, and what would be good and bad arguments around that. But I've never suggested anyone buy a token - indeed, I was pretty vocal in pointing to speculative bubbles and silly ideas, like NFTs.
Crypto today has a lot in common with both the internet in 1993 and the internet in 1999. Huge potential with few of the use cases invented yet, combined with froth, scams and delusion. This makes it easier to dismiss (“useless AND a scam!”).
But dismissing crypto as a useless scam is much like looking at Usenet, Cuecat and Boo .com and dismissing the internet. It mistakes applications for the enabling layer.
Looking at crypto and only seeing the scams is like looking at the internet in 1999 and only seeing the bubble.
Looking at crypto and seeing no use cases is like looking at the internet in 1993, when the web was 3% of traffic
Another parallel:
1993 - people complaining the term should be internets, not internet
2018 - people complaining 'that's not what crypto means'
You could argue I was wrong and blockchain's potential never turned into anything much. It actually has become a huge deal as plumbing in the finance industry, but not much else. But so what? This was an interesting tech that hasn't really worked out. Welcome to the tech industry.
I dunno... but those self-quotes sound a lot like at least implying that "missing out on crypto" would be like missing out on early days of Internet. You also seem to try and retrofit "blockchain" instead of "crypto", but these are two different things - blockchain is a technology, crypto is a form of play-money based on that technology, just like it's older brother, the monopoly-money, is based on the technology of paper (but not much else).
No, I was using crypto and blockchain as interchangeable terms. I'm aware some people don't, but that's always how any person the space I've spoken to uses them
I don't think it's fair to consider writing an analysis (even a favorable one) about a topic as _shilling_ for it. It's not like he was pushing shitcoins or minting NFTs.
I have always been (and remain!) bearish on crypto but it absolutely was something that couldn't be ignored a few years ago. Even if you came to the conclusion that it was bunk, there was significant enough fervor that any technologist needed to reckon with their position on it.
For example, lots of engineers proclaimed very loudly that document databases would replace the RDBMs, or that GraphQL was the future of APIs. They were wrong, as it turned out, but only with the wisdom of hindsight.
No, I didn’t. I made my account private and stopped using twitter.
And the actual tweet says the opposite - that NFTs would need to develop some kind of cultural grounding for them to become a investment, which they didn’t have at the time and never got, and without that this would just be a speculative bubble, which is exactly what happened.
I made it very clear that I thought NFTs were a speculative bubble. I never suggested anyone should buy any crypto-related instrument. The idea that I was ‘shilling for crypto’ is something you would only say if you’re an idiot, as the OP and a few others on this thread clearly are.
What's wrong with that? There are now materials that allow you to have solar panels on a window (so they are not opaque anymore), and we can put data centers under our feet.
Ok I guess my mind didn't think "entire surface of the earth" would really mean entire surface of the earth because that's just abhorrently stupid. I mean, in order to cover the entire surface of the earth with datacenters, we would first need to build stuff on the entire surface of the earth... Which would take... thousands of years? And I'm sure we would grow some trees on top of the data-centers along the way.
In the current system technology is meant to serve the shareholders. That might end up serving us, if you believe the standard narrative. But maybe the shareholders will just carve out a few nature reserves for themselves and just wait for the rest of us to die off.
Here's a test to know if this is/will be true: look for a situation where the "needs" of AI (e.g. land, electricity, etc) conflict with the needs of people (e.g. land to live on, grow food on, electricity to light our homes).
Find a place where the needs of AI conflict with people, and observe who wins out.
Does the entity that owns the datacenter say, "oh sorry! I guess we're using too much electricity. No worries! We'll stop doing that" ...or does it say, "lol too bad, all the electricity belongs to us!"
Does the entity wanting to build a datacenter say, "oh sorry! We thought you'd be okay with us using this land. But if you're not that's okay, we wont build here" ...or does it say, "lol too bad, we own the government and they're seizing the land under eminent domain!"
Who made Ilya Sutskever, or any other LLM-bonehead the Grand Prophet of Humanity? Why the fuck is his opinion on that relevant? Of course he will shill for data centers.
I'm old so my computer career has gone: punch cards => calculators => command-line => GUI => touch screen => voice => chat. Chat seems to be the best blend of expressiveness and utility, with a dose of magic thrown in.
> "What happened the last time that everything changed?"
Honestly, I'm glad we hear more of the commoditization of AI, and I hope that the comparison of AI with water or electricity will become mainstream and that the states (as in nation states) will understand that sooner rather than later and act accordingly.
Ben, I follow you and think you’re brilliant, but boy you can’t take feedback like ever.
Duplicate slide is there clearly to add the question, but it has a glaring typo—- “predications” instead of “predictions”.
A random internet stranger read to page 51, which is already a rare occurrence, and helped you find a typo that you can now edit.
But sure, the answer to that is “your comment is wrong”.
Ah, that’s different. Your question confused me because you said they were duplicate slides and they are not, and your word ‘predication’ looked like a typo by you.
The slides say "predication" and not "prediction." Just a typo :) I think the AI slop comment was excessive, and I liked your deck, but there is indeed a typo you could fix!
You can find the 4 versions of Benedict's deck here: https://www.ben-evans.com/presentations I appreciate the temporal view into this thinking. My interpretation:
Nov 2024: Don’t dismiss this; it may be the next platform shift. But the actual questions are still unsettled: scaling, usefulness, deployment, and business model.
May 2025: The model layer is already showing signs of commoditization, so the important question shifts toward deployment: products, use cases, UX, errors, and enterprise adoption.
Nov 2025: The capital cycle has become the story: everyone is spending because missing the platform shift is worse than overbuilding, but there is still no clarity on product shape, moats, or value capture. That creates bubble-like dynamics.
May 2026: Provisional thesis: models look likely to become infrastructure, while value probably moves up-stack into apps, workflows, product, proprietary data/context, GTM, and new questions made possible by cheap automation. But he is still explicitly calling this provisional.
Thanks for the summary. I do love Benedict‘s work; I find he’s one of the few commentators who consistently strikes a balance between taking the transformative potential of AI seriously while not falling over into hype.
Some things that stand out:
* He’s really good with his historical analogies, especially looking at previous transformations like the early Internet and mobile; no surprise given that he has a history degree.
* he emphasizes over and over how we have still have no idea how all of this is going to work when the dust settles. I think that’s kind of a historian’s move as well. When you look at what people were saying during the early days of the web, for example, almost all of their predictions weren’t just wrong… in hindsight, given how the future played out, they were asking the wrong questions. The implication is that we are probably asking the wrong questions about AI too.
* Nonetheless his thesis about the commoditization of models is actually a fairly strong concrete prediction. i’m not sure if I agree with it entirely, but I do keep it in mind every time I look at the valuation of leading AI labs.
* he continually makes the point that a chat bot is barely a product and that AI labs have so far had very little success in delivering products above that layer… with the exception of coding agents, of course.
I just got a bit triggered by the "hype" word. What if the hype was real? It is easy to say that nobody knows how all of this is going to work, and I would say it is a prudent thing to say, but there is value in making a bold prediction from the start instead of just updating your view to respond to change. In one case you are predicting stuff, in the other, just reacting.
But I absolutely agree that in hindsight we are often asking the wrong questions about each new technology.
I keep seeing on HN that AI is a hype, and many here are anti AI (which I get, as a programmer AI made my job less interesting, and I'm even worried about losing it), but where has AI underdelivered?
The hype is in what AI delivers (at least so far). I would never create a PR without an AI review. I will ask an AI to write code for me from time to time.
But it still has huge gaps in quality. And from time to time, it shows me that it doesn’t really understand things. You might point out that how is that any different from your mediocre engineer. But for most people skilled enough, you can easily know the difference when someone doesn’t really know something.
With AI, you discover this after reading several pages being dumped on you by people being “more productive” with AI.
Ok so the hype would be people saying AI can currently do something well and autonomously when it cannot (or not consistently enough), and it is easy to prove them wrong.
But I feel like people are more hyped about what the AI will be able to do soon rather than what it can do now.
I think AI does understand things (depending on your definition), how else could we communicate and ask it a question if it didn't? I mean we're quite far from Eliza here.
And yes, often their answer would be so wrong that we think it is impossible that AI understands anything, but this jagged intelligence doesn't prove, at least to me, that there isn't some understanding. At what point do we say that AI understands things? What if we can reduce 99% of those dumb failures, would we then say than AI understands?
>I think AI does understand things (depending on your definition), how else could we communicate and ask it a question if it didn't?
https://en.wikipedia.org/wiki/Chinese_room
That doesn't really respond to the question though - there is a quite reasonable argument that the Chinese room as a system 'understands' things.
The issue that is hit immediately is we don't have a definition or test of understanding that AI doesn't clear easily. Then on top of that we can't even really be sure that we ourselves are understand things given all the tricks that our minds play with memory and perception. There is precious little evidence that the people around us understand things, they seem to be guessing. It is completely unclear if a Chinese room has or doesn't have a property if we rule out all the tests that check for it as not really counting. But all the tests we can do suggest it does understand, because engineers can implement Chinese rooms now and they even turn out to be more reliably artistic/capable of novel thinking/creative than humans. Anything that tests understanding they can do.
> and it is easy to prove them wrong
No, they just say you are using the wrong model or something.
If it's a coworker dumping reviews of crap code on you at work, the incentive is to blanket approve everything because otherwise you're just the grumpy old man who is resisting innovation. No matter that the code makes no sense at all and the tests aren't actually testing what they should test.
>where has AI underdelivered?
Other than the stock market (which seems decoupled from reality at the moment), where has AI delivered?
The only use case where I see anything resembling AI delivering on it's promises is software, and my personal experience with that is that everything that comes out of the teams using AI is destructively broken. (Where they used to be able to deliver software that worked, even if it wasn't ideal, now they reliably make things worse and their stuff doesn't work when used.)
I agree, especially the juxtaposition of "we have still have no idea how all of this is going to work when the dust settles" and "hype". If we don't know, then there is a chance it isn't a hype.
For example, now it may seem that the models are becoming mere infrastructure, and the value moves up to apps and data. But if the models of tomorrow become able to write the apps themselves, then the value moves back. I won't need to pay some to write me a wrapper for the LLM, if the LLM will be able to write the same wrapper, maybe even better because it will be customized for my needs. The app providers are currently profiting from the gap between "what a software company can do using the AI" and "what the AI can do unaided", but that gap is going to shrink, possibly to zero.
> for example, almost all of their predictions weren’t just wrong… in hindsight, given how the future played out, they were asking the wrong question
Do you have an example of this? My (poor) memory remembers "it's going to change how people buy things", was the big deal at the time, and it seems like it was a great prediction.
Well, yes, but as the other commenter says, that’s a very broad general statement akin to something like “AI will change knowledge work“. That’s certainly true, but how? What are the details? What kind of companies are going to be the winners and what kind will be losers, or end up with commodity margins, like the telcos did after the mobile revolution? What is the pricing structure going to look like?
I suppose a concrete example in 1997 would be that a lot of companies thought the future of e-commerce was setting up a store on AOL, that people would use while sitting down at a desktop PC. Obviously it didn’t turn out quite that way. Furthermore, the Internet enabled new kinds of ways to buy things that weren’t even envisioned in the pre-Internet pre-smartphone world: think Airbnb and Uber.
Predictions are hard, especially about the future. Most predictions reflect the worldview and biases of the time in which they are made: think about all the vintage sci-fi from the 60s 70s and 80s that actually reads or looks kind of retro now. Similarly, our predictions of the future will look kind of retro and strange to someone living in the 2030s or 2040s. If studying history has any lesson to teach us, it’s really just this: that the past is an alien world with alien moods of thinking, and that our moment in time will look similarly alien to people in the future who choose to look back and analyze it closely.
This isn’t an argument that we should stop trying to make predictions. We need to, but it is an argument for humility, and also for questioning all your assumptions that you might be importing.
That's a very vague prediction that took decades to bear fruit. The concrete predictions behind the investments into companies like Pets.com and Webvan failed. It took the survivors like Ebay/Paypal and Amazon to build the digital payment and shipping infrastructure over decades until cultural acceptance hit critical mass.
Agreed, I appreciate his historical perspective, but I think one critical mistake his posts make is implying, largely because the parallels to history have been similar so far, that history will repeat.
Like, yes, the telecom bubble was a clear case of overbuilding and the AI data center "bubble" looks a lot like that... but this overlooks that the fiber capacity being laid back then far outstripped the demand, whereas all the compute providers today have been desperately crunched for capacity, despite investing almost a trillion in CapEx -- to the tune of almost a trillion dollars more of backlog -- for multiple quarters now.
Or yes, historically new technology has always created new jobs... but all those new jobs required a higher skill level along dimensions that current AI models are already good at, meaning we've never had a technological revolution quite like this.
Or yes, prior technological revolutions consigned incumbents to irrelevancy, primarily due to shifts in technical platforms... but then today's business leaders are 1) very well educated about what happened to their predecessors, 2) very paranoid about the same thing happening to them, and hence 3) are actively making moves to capitalize on the next platform shift.
I also think his dismissal of chatbots is a bit premature. It is precisely because chatbots operate via an extremely simple, flexible and natural modality, i.e. a conversation -- entirely unconstrained by the form factor necessitated by any app -- that their infinite use-cases have become unleashed.
My take is that the AI labs are actively exploiting this extreme flexibility to surface valuable use-cases -- one of the hardest parts of innovation -- at which point they can simply slap an agent on top of them. Which is, yet again, simply a chatbot, except one that can actually do useful things for you and hence can be charged for a lot more money.
I didn’t make any comparison at all with the fibre bubble, for precisely that reason. The comparison is with mobile data, which was and is always behind capacity.
I think one of the things that the usage data shows us is that chatbots absolutely do not have infinite use cases - most users only use them a day or two a week or less.
That's fair, I may be conflating your takes on mobile data with others who've made the comparison to the telecom bubble, and if so, mea culpa!
But I also do disagree with the take that usage patterns indicate a fundamental shortage of use-cases. Yes, everyone reports WAU instead of DAU because WAU numbers look much more impressive, but I think the extreme shortage of compute plays a major role in this. I suspect all the AI labs are deliberately holding back from pushing AI adoption too much because of this. (Google execs have even made comments internally to this effect.) Note that even at such low frequency of usage all the model providers are desperately strapped for compute, which means there is insanely high demand from some quarters.
One way how capacity limitations could impact adoption is that the free-tier models are not as good as the frontier ones, so the free users come away less impressed with AI capabilities, leading to lower regular usage. This problem is larger than it appears, because it can take a long time to figure out how to get AI to work for your use-case, and people simply have not experimented nearly enough, partially due to first impressions. On the other hand, most companies seem to be OK with huge tokenmaxxing bills!
It seems to me the AI players are all playing a delicate balancing game across three fundamental dimensions: adoption, monetization, capacity. That is, they are simultaneously 1) pushing free / cheap AI usage as much as possible to hook users, capture market share and suss out new use-cases, while 2) carefully allocating token quotas for the most lucrative use-cases to satisfy investors, and 3) balancing available compute between those two competing priorities. I suspect as the compute bottleneck is alleviated and frontier models become more accessible cheaply, we'll see way higher DAU numbers.
> new technology has always created new jobs... but all those new jobs required a higher skill level
The industrial revolution didn't seem to require any particular special skill at all. Just anyone who was willing to tend to a machine all day. (Maybe that's a parallel...)
You're right actually, what I really meant to say is a "higher-level skill" rather than a "higher skill-level." Higher-level skills don't necessarily mean a more difficult skill, they're usually just at a higher level of abstraction.
Specifically, the 3 dimensions along with new jobs required new skills were: a) cognitive, b) technical or c) social skills. I guess tending to a machine was a mix of a) and b), because even if the controls were straightforward, it probably required some understanding of the underlying mechanisms.
I think that DeepSeek may be important to that. They have a really good model that's open source, raising the bar for all other players: how good your model needs to be so you can make meaningful money on it (better than DeepSeek).
Same thing happened on other places the open source offering became popular.
I think the original DeepSeek moment seemed important. And yes, the more recent model is good, but there are multiple. This commodification trend spans many different companies, including Kimi 2.5/2.6 and GLM5.1, and even Google itself with its Gemma models. There are a dozen models that exist at roughly the frontier from 6 months ago at 1/10th the cost.
> that exist at roughly the frontier
no disagree, specifics matter.. There are a dozen well-defined LLM application subject areas that are regularly tested.. one overall grade IMO lacks important detail.. To go a bit abstract, it is ironic that "oversimplification" in the discussion of these complex machines mirrors the effects on information of the automations themselves.. constantly simplifying, substituting and diluting real meaning
What good is an open-weights DeepSeek model if you have nowhere to run it?
OpenAI / Google / Anthropic / XAI also have a ton of compute. That is the real moat.
It's quite expensive to self-host but you have many places to run it. OpenRouter alone lists a dozen different providers for DeepSeek 4 Pro. https://openrouter.ai/deepseek/deepseek-v4-pro/providers.
So long as there is demand, there are always going to be providers competing to offer it at a low cost. My understanding is that the median price on there is in the ballpark of what it costs to run the inference. This is very different from e.g. Opus, which you can basically only buy from Anthropic at the price they set.
antirez running (quantized) DeepSeek V4 Pro on a Mac Studio M3 Ultra with 512GB of RAM:
https://bsky.app/profile/antirez.bsky.social/post/3mlzwmvlov...
It's much closer than you think. We're going to see specialized hardware in the next 24 months capable of running 2025-era frontier models. That's big.
2-bit quantization? That's a lot of signal being removed. Considering how quickly the AI models are progressing in their capabilities (still exponential curve), I will not want to use the 2025 model in two years time. Similarly, how I don't want to use llama-3 or old Anthropic model from 2023 or 2024. Newer models are so much better that it makes it very difficult to ignore.
Once and if the advancements with the AI models slow down, only then IMHO it will become feasible to design the specialized HW for general-purpose consumption and general-purpose workloads.
Opus 4.6 was a 2025 model and many people (myself included) feel that if that's where models peaked, we won't be disappointed.
Even at 2-bit quantization, DS4 is probably on par with a 2024 frontier model. You can run that today on local hardware, and at a minimum, local models are going to keep pace over the next 12-24 months. Even if they don't close the gap with frontier models, they'll still play an important role in the overall pipeline for cost, speed and privacy reasons.
That's without even mentioning the additional capability that something like a Taalas chip churning out 17k tokens/sec could unlock.
It's big because it may take a big swath of people who will actually pay for LLMs out of the market. But for the average consumer they're going to primarily use their phone/tablet and we're far away from that being possible.
Even if it were possible the LLMs are such a gold mine of user data. It's really hard to see that opportunity be passed up.
That specialized hardware will be scooped up by AI data-centers, just like RAM is today.
No more than Mac Studios. Datacenters need different hardware.
The 512 GB ram studio can't even be purchased anymore. It's been delisted
https://www.apple.com/shop/buy-mac/mac-studio
Same with the Mac mini. entirely removed from all store references
I just got into self hosting Deepseek v4 Flash on a single DGX Spark via antirez’s DwarfStar 4 project
It feels great to finally have access to something local.
That seems pretty temporary if people can just build more compute.
There are myriad compute providers. I suspect the inference market is hard to monopolize. But given our anti-trust track record the past 40 years I suppose it’s possible.
Well, yes. Anyone who tells you they know how this is going to work is an idiot.
I didn’t know there were a sequence of these decks; thanks — it’s helpful to think of them as updating snapshots in time.
The main thing that stands out to me on these graphs is just . how . early we still are - looking at industries like legal which in my mind are certainly going to be massively disrupted, and seeing the very low usage rates vs. tech (which still shows less than a quarter of tech people using AI daily) — we are in for a lot more change than we’ve seen so far.
Legal has lots of institutional inertia behind it though. I think AI will be very very useful for lawyers..... at their desk in private. But I don't see it replacing them. The legal system is heavily personal and relies a lot on reputation and tradition. I think you'll see courts, bar organizations, etc frowning on using AI too heavily, and certainly not using it to automate "official" processes.
I appreciate Evans’ work and wrote an “antithesis” to the Nov 2024 iteration of this. Given the pivot to “models look likely to become infrastructure” I might want to update my take.
Didn't you mean Claude take? It's ai written after all...
This is a reasonably well-examined take of the situation.
On the technical side, one of the additional things I've had on my mind is the potential that these mega models are in fact hiding a ton of inefficiency.
The approach of simply shoving higher dimensionality and more parameters into largely tweaks to the current models has delivered results, but it feels like "mainframe" era of computing to me.
Throwing reams of annotated human content and forcing the machine to globally draw associations from it feels clumsy. Just as people are able to learn structured knowledge via rule-systems that are successively elaborated with extensions and situational contradictions, I feel like there's probably a much more compact representational model that can be reached by adapting the current technical foundations (transformers, attention, etc.) to work well with generated examples from rule-systems, that then gets used as a base layer to augment the "high level" models that process unstructured data.
The risk for the behemoth datacenter might be similar to the risk in the early computing era of building compute centers right before the PC revolution took off.
If it turns out that there exists some more compact and efficient representation for this intelligence (which IMHO is likely given that we are still in the first generation of this technology), the datacenters may end up decaying mausoleums of old tech that has no relevance to a distributed intelligence future.
That's the big technical unknown unknown for me. How much efficiency juice is there left to squeeze, and what does that mean for a distributed landscape vs a centralized datacenter based landscape.
Right, the crazy thing is that much of the groundwork for the “rules-and-heuristics” mode of AI was laid down in the 70s and 80s, long before we had the raw compute power to reliably extract patterns from reality-scale inputs. Those early efforts failed miserably mostly because the rules had to be populated manually and in a ridiculously space-inefficient format (compared to the density of information in model weights).
So yeah, the next stage is models that basically do what humans do: encode causal models of the world in a composable, symbolic form that can be falsified and refined through interventional experiments.
> much of the groundwork for the “rules-and-heuristics” mode of AI was laid down in the 70s and 80s, long before we had the raw compute power to reliably extract patterns from reality-scale inputs. Those early efforts failed miserably
Yes, and: we concluded that enough of reality doesn't work like that. The formal reasoning space is very powerful, but all the stuff we're really interested in has enough ambiguity and generalisation in that you can't cover it with a "small" set of rules.
Maybe if you had a really large number of rules? And used matrix multiplication to make sure that you covered all the marginal interactions between every possible set of rules? And then had some means of looking back on both output and input to constrain it towards things that were relevant? Wait a minute ...
I feel like the talk about "world models" is trying to reach at that, but cast it in different terminology. World model is just domain model, and once you're at domain model, there are multitudes of domains.
Unsupervised learning over domain rulesystems has the potential to let us define really well-defined, scoped models that behave a lot more deterministically and don't colour outside the lines, and reserve their weights for cleanly modeling the domain associations and relationships that matter.
I just asked codex the following question in the middle of my coding prompt:
Answer: As amusing as this may be, I really have no need or desire for my coding model to understand or be aware of ewoks and their relative strengths compared to jawans. Nor do I need it to understand the nuances of the races of middle earth. And prompt response of "I have no idea what you are talking about" to all of these would feel reassuringly scoped.Mixture-of-Experts seems like an attempt to do this - the domain structure being extracted into specific sub-models that are presumably trained on particular domain-associated content - but it feels like this is once again the beginnings of what is possible.
> As amusing as this may be, I really have no need or desire for my coding model to understand or be aware of ewoks
You'll think otherwise the first time you're a victim of a zero-day ewok.
Seriously though, while coding models may not need to know about ewoks, their contextual knowledge of things beyond just writing code almost certainly makes them better coding models.
It could be difficult to constrain the training corpus "just right" so that you eliminate all the irrelevant subjects like ewoks but retain enough so that the model doesn't turn into an idiot savant capable of churning out correct code but incapable of understanding what you really want.
I've been having similar thoughts, regarding the gigantic trillion parameter models. I'm starting to believe the future will be very specialized focused models thant can be run on modest hardware (locally) but that can scale in performance (latency, speed) in the cloud, much like any other software of today.
If you need to do programming do we really need trillions sized models? Other domains might be large or smaller, but there's no need for a model to 'know' everything and datacenter levels of hardware to run.
General chatbots might work better as larger models since you really don't know what people will also for, or alternatively we find a way to route the initial question to the appropriate model. Like MoE but without needing to load a gigantic model into memory first.
> Mixture-of-Experts seems like an attempt to do this - the domain structure being extracted into specific sub-models that are presumably trained on particular domain-associated content
This is a common miss-conception. MoE LLMs are NOT trained with each expert receiving domain-associated data. It's just an unfortunate naming decision that stuck, and is commonly miss-understood by non practitioners.
Interesting. So what's the strategy there? Just assume that each expert will learn some underlying clustering of semantic associations, but not direct it?
Not even that. The "experts" are not expert in any particular topic.
MoE is an architecture change meant to lower the total compute for both training and serving an LLM. You basically have many smaller models (unfortunately called experts) and a router on top of them. The router "learns" which expert to activate for the next token generation, but that doesn't need to follow any semantic association. For the same math problem you could get experts 1 and 234 activate on the first token, 5 and 132 on the 2nd token and so on.
> Imagine asking “What will be changed by the internet?” in 1997
Pretty much all of the stuff that was suggested back then or earlier: Shopping, advertising, video conferencing, collaboration, software distribution, media consumption, banking, finance and of course communication overall.
Most of these ideas weren't exactly new in 1997, but go back to services like CompuServe and even Douglas Engelbart's Mother of All Demos. The bottlenecks were bandwidth and personal computer performance (both of which were then predictably following Moore's law), not human imagination.
A few examples that a lot of people correctly extrapolated from: NLS (1968), PictureTel (1987) and later LiveShare, IndyCam (1993), CUSeeMee (1995), RealAudio (1995), RealVideo (1997).
Perhaps the core business problem with LLM:s isn't finding a product-market fit, but that our imaginations have been running wild with expectations on "AI" since at least the 1950s, and now we have something that quacks - but doesn't quite walk - like a duck.
Knowing why we're trying to build something is a good smell test to segregate promising tech from snake oil, in my experience.
Take quantum computers for example, a lot of the time people will compare that to the dawn of classical computing, with claims such as "we can't know yet what we'll be able to achieve, we have to build it first!". Except that even the first classical computers were built with goals and applications in mind. Turing's was to decrypt Nazi codes, for example. Instead, when asking a quantum computing company what they're trying to achieve, they'll gesture vaguely at "chemistry, finance, ecology".
I think a more nuanced take is appropriate here. It‘s true that computers were invented with the express goal of speeding up military and corporate computing (back when computers were still people), but their influence on our culture and society extended far beyond those initial applications. The telephone was invented as a means of long-distance communication, but it shaped our values surrounding communication as well. Therefore it may be hard to predict what will ultimately become of a technology.
I agree that there are a lot of overhyped technologies though. Quantum computing has been in the works for decades now, with little to show for it in the popular perception.
> Except that even the first classical computers were built with goals and applications in mind. [...] Instead, when asking a quantum computing company what they're trying to achieve, they'll gesture vaguely at "chemistry, finance, ecology".
I think the problem is a little bit more subtle:
To finance a lot of innovations, better also some intermediate step towards the far goal should already be very useful, otherwise the company that builds it will go bankrupt.
If this is not the case, it's typically not commercially viable, some product category is typically basic research (which is very important, but it typically means that the commercial potential will only come up in some future).
There do exist problems where a quantum computer gives an extreme advantage in the sense that we have no idea how a fast classical algorithm could look like. So, the only viable approaches for these problems are:
1. work on a huge algorithmic breakthrough (to be able to solve these problems fast on a classical computer)
2. build a quantum computer
What are these problems?
They are basically all special cases of the abelian hidden subgroup problem:
> https://en.wikipedia.org/w/index.php?title=Hidden_subgroup_p...
In particular cf. the table at the end of this Wikipedia article:
> https://en.wikipedia.org/w/index.php?title=Hidden_subgroup_p...
If you do have such a problem to solve, 1 and 2 are the only viable approaches.
So, there do exist goals and applications for which a quantum computer is insanely useful (assuming no huge algorithmic breakthrough happens).
The questions are thus:
- Is the abelian hidden subgroup problem sufficient for being able to carry a whole potential industry?
- (To come back to my introduction) What use does a quantum computer that is only capable of solving very small instances of this problem have for the user?
I am familiar with these applications. Indeed, your questions are very relevant. I've personally decided a long time ago that no, these applications are insufficiently useful to justify the billions invested in quantum computing, and the billions more that will be required to build anything remotely capable. So far, nothing's come up to make me reconsider.
In slide 22, it compares LLM labs (OpenAI/Anthropic) to mobile data telecoms (AT&T, Verizon, TMobile) in 2010s. The difference is that mobile telecoms follow a standard (3G, 4G LTE, 5G) and there is little to no differentiation. It's virtually the same no matter which company you choose or which country you travel to.
A better comparison is actually AWS/Azure/Google Cloud/NeoClouds to AT&T and Verizon. The data centers follow a standard (CUDA/PyTorch/etc.) while OpenAI and Anthropic are becoming more like iOS and Android. Both the clouds and telecoms had to spend a ton of capex to build out infrastructure first.
Because of what I think is a poor comparison, the the next few slides make the wrong conclusions. For example, it thinks that models will be a commodity like 5G data. I disagree. I think frontier models are a classic duopoly/monopoly scenario. The smarter the model, the more it gets used, the more revenue it generates, the more compute the company can buy, the smarter the next model and so on. It's a flywheel effect. This is similar to advanced chip nodes like TSMC where your current node has to make enough money to pay for the next node. TSMC owns something like 95%+ of all of the most advanced node market. Back in the 80s and 90s, you had dozens of chip fab companies. Today, there are only 3. There should only be 1 but national security saved Intel and Samsung fabs.
There is evidence that the Chinese models are falling further behind, not gaining. Consolidation will likely happen soon because many unprofitable open source labs will have to merge and focus on revenue generation.
I've made the semi comparison myself, but the amount of capital required to build a SOTA model today is clearly nowhere near enough to lead to a monopoly.
I'm aware that telecoms networks are standardised (I was once a telecoms analyst), but that isn't a precondition for a commodity.
Just like how starting a chip fab was relatively easy back in the 80s and 90s. There were dozens of chip fab companies in the 80s.
It turns out that fabs follow Rock's Law which is that the capital cost to build a new fab doubles every 4 years. This means it will quickly get rid of the less competitive players. This is not dissimilar to the LLM scaling laws where you need a magnitude more compute to get unlock a new tier of intelligence.
Today, Anthropic and OpenAI are clearly in the lead for models and then there is everyone else. Google is a close 3rd. No one else is challenging them anymore in SOTA models. Some models might beat them in one or two benchmarks but none can compete overall. I expect this gap to grow bigger as models cost more and more to train.
I doubt it. Compute costs tend to crater over time, and LLMs will almost certainly plateau. So the opposite is almost certainly the case: over time, it becomes cheaper to train.
By that time, there would be a clear monopoly or duopoly that controls the revenue, distribution, and compute.
Possible, but only if there’s regulatory capture. Which will be difficult since any country could produce a SOTA model that everyone can use for free
Yes, I wrote about Rock’s Law too, but we don’t know that this is how these models will develop
Evidence point to the same type of scaling law. Compute for a training run grows 4-5x every year.[0] I'm sure this will slow down but the premise remains that weaker competitors will not be able to maintain this pace. We already see labs like Cohere, Mistral, Inflection AI, Adept, Character.ai, and others bow out of the frontier race. I'm also skeptical that Meta, xAI can catch up. Even Google has trouble keeping up.
Even if this isn't true, comparing telecom bits to tokens is wrong. Bits are the same no matter what telecom transfers them. Tokens are not all the same. The quality varies.
We're already seeing a massive divide between frontier models and lesser models in growth rates. Anthropic is adding $10b - $15b every month in ARR. This figure likely dwarfs open source labs. This is all because its models are maybe 10-15% better.
The cost to inference a 1T param frontier model is the same as a 1T param open source model. Therefore, if the frontier model is even 10-15% better, it will gobble up the market over time.
Lastly, even though Claude Code and Codex are the biggest revenue drivers for Anthropic and OpenAI today, I don't believe this will be true 2-5 year from now. I believe selling their tokens via API will be their biggest. The sum of applications in the world will dwarf coding in market size. For example, biotech, finance, physics, engineering, robotics, sensor data, etc. This is why I think OpenAI and Anthropic are becoming more like iOS and Android than AT&T and Verizon. Applications will build on top of OpenAI and Anthropic just like iOS and Android.
[0]https://epoch.ai/blog/training-compute-of-frontier-ai-models...
How about the externalized intelligence around the model weights (skills, tools, harness, memory etc)? If the model weights are sufficiently intelligent, the focus might move to the external layers.
I agree with much of what you’ve written but think you are missing the correct alignment of the mobile data timeline — mobile data had standards because it was forced to. It was forced to early because it was not a fundamental innovation, telecom itself was the fundamental innovation, mobile was a constraint relaxation. Intelligence might be forced to have standards as well, we will see what form the regulations take when prices reflect costs and healthy margins and become existential threats for many businesses.
Intelligence can’t be standardized.
The reason mobile data had to standardize is because it’s a network and a network must have protocols. It’s useless without them.
I agree with that as a premise, but again it seems to me you are selectively jumping way into the end game. There were early networks that did not standardize, and these nonstandard networks had advantages, and some of those advantages were sacrificed in market-driven standardization.
Intelligence must have interfaces, and those can be standardized. Businesses will try to remain provider agnostic, which will also drive standardization via standard sales and marketing methods.
Separately, we are doing our best to standardize performances on benchmarks.
I don’t disagree that right now transport of standardized mobile data vs emulation of human intelligence is qualitatively different, but perhaps primarily because it is early in development, and our vantage point this time is relatively from within the network, instead of outside it.
You lay out some good arguments but I agree with both: the models relative to few years back really did become the commodity because today you could take the non-frontier model, maybe self-host it or pay the much less price per M tokens to get the performance of a ~2-year old frontier model. At the same time I do think that we are getting into the monopoly/duopoly/tripoly with the frontier models for all the reasons you already mentioned, and this scares me a little bit.
Lower intelligence LLMs can be a commodity, yes. But these won't make much money, if at all. At the end of the day, it costs the same to inference a 1T frontier model and a 1T free model.
OpenAI and Anthropic don't compete in the LLM commodity market. Hence, I had a problem with slide 22.
Most of your analysis I can easily relate to except “There is evidence that the Chinese models are falling further behind, not gaining.” Where is that evidence? Deepseekv4 claims to be trailing front runners by six months. I read people agreeing with this. I watched Eric Schmidt to recently make similar comments. Is he just scaremongering? Why do you claim they are falling behind?
A few evidence I pieced together:
1. Ex-ByteDance Engineer said so recently: https://www.businessinsider.com/ex-bytedance-engineer-says-c...
2. A benchmark concludes so: https://www.nist.gov/news-events/news/2026/05/caisi-evaluati...
3. Generally, people say Chinese models bench better than they perform
I think it seems to make sense given that China does not have access to Blackwell while in the past, they at least had access to gimped H200s.
> What happened the last time that everything changed?
* Hardware era (pre 1995s) -> IBM, Intel, Microsoft, Apple
* Internet era (1994-2001) -> Amazon, Google, Meta, Salesforce
* Mobile era (iPhone+ era) -> Uber, Mobile Games, Youtube, Snapchat, Tiktok, Airbnb
* Cloud era (AWS+ era) -> AWS, GCP, Azure, Snowflake, Databricks and bunch of other data & database startups
AI era (ChatGPT+ era) -> Change is inevitable
Nice breakdown! I would separate the Hardware era between Mainframe era and PC era. I would extend Internet era a bit more, Perhaps 2007 when the IPhone was released.
Edit: I hadn't seen the original presentation yet. I see that Evans already divided the eras like I suggest above.
? That appears to be arbitrary eras then arbitrary companies from that era. Do you think Amazon and Google disappeared after 2001? Do you think databricks is now bigger than IBM?
Change might be inevitable, but I'm not sure your list shows or proves that.
What I wanted to say is every era gave birth to something big.
AI era will get its own winners, but there will be some new big players as a result of this era I think
> Do you think Amazon and Google disappeared after 2001?
I don't think that was implied at all, just that the context of the web is what allowed those companies to pop up.
With each platform shift, some of the old players disappear and some of them become irrelevant - IBM is still with us but no one cares
> * Internet era (1994-2001) -> Amazon, Google, Meta, Salesforce
Meta, née Facebook, wasn’t started until 2004.
[dead]
[dead]
Lots of quotes from Mark Zuckerberg, not a lot of Zuckerberg quotes on the $80b invested in the metaverse
I had that exact chart in a previous presentation.
I was a baby when the Internet Revolution happened. I was in high school and college when the Mobile Revolution steamrolled everything. It’s been interesting to see this one, as an adult working in the world. I wonder how far it will go.
Further than the doomers think, but not enough to pay off the investors of the original boom. I say that as someone who has been an early believer in the internet (first website in the 90s), mobile data (slurping down the 'net, IRC, and IMs via EDGE data), smartphones (N80ie), streaming media (RIP Windows MCE), the list goes on.
Models were always going to be the commodity, just like the most popular and viable use cases at present are less job-replacement than "let's analyze huge data sets for patterns we're missing, and adjust accordingly" or "probabilistically generate deterministic software for me for X function/task". One-offs simply aren't profitable when models are interchangeable commodities, hence that brief attempt to pivot to "pay by outcome" before giddily embracing the classic consumption-based-billing playbook.
> Further than the doomers think, but not enough to pay off the investors of the original boom
Not an uncommon event - not only did this happen to many companies who were big in the original internet boom (e.g. Sun Microsystems, as well as all the Boo, Pets.com etc), it also happened to the railway boom of the previous century, and even the Channel Tunnel.
If coding is such a big part of LLM agents' usage at the moment, I do not understand how far the best models will continue to shine and take the largest chunk of revenue. I am far away from tech hubs but I think better harness will utilize smaller models for more constrained, efficient and reliable coding agents.
In a way this is like distilling (but it is not) but you can make better harness (tackle more edge cases, better tool/function definitions, sandbox handling, bash management, DB management, deployment management, etc.) but extracting what LLMs know into code.
Maybe I am wrong but I would like to see custom software for the last mile (tiny/small businesses) becoming a reality. AI would eat the world of software but costs would go down since you can extract value upstream from the LLMs and spread downstream through tighter coding agents.
I am building a coding agent that will not be small - it will be a lot of code, carefully mixed roles (mimic a software dev shop) with separate tools available to different roles. And all this code is generated by other coding agents. https://github.com/brainless/nocodo
I am a nobody from nowhere with 18 years of software engineering behind me. I do not care about revenue. I just want to see a regular business owner's workflow going live on their own VPS.
excellent work！
Wait. There were 10000 elevator attendants in the USA in 1990?
Supposedly, in 1990, there was somewhere between 132,000 and 270,000 travel agents. Consider that.
>"We see a future where intelligence is autility like electricity or water and people buy it from us on a meter” Sam Altman
I always considered jokingly that I am "selling" my intelligence when I work for a company. This clarifies that my perception wasn't far off.
Hopefully you are renting your intelligence to the company, not selling it.
(Unless the job gets you burned out, in which case it was selling indeed.)
>>Companies report ‘annualised’ revenue, defined as sum of previous 4 weeks multiplied by 13.
why is it multiplied by 13?
In business there's 52 (4*13) weeks in a year and as a result, 2080 regular working hours in a year (40*52). I think these are just generally agreed upon ways to define time for simplicity. In some (most?) systems your 'hourly wage' is simply your salary divided by 2080, trying to divide your salary by other metrics to determine hourly wage tend to wonk the numbers a bit.
this took a bit of a mathematical turn because of my poor phrasing. what i was actually intrigued by was how does revenue of 4 weeks become "annualized" by just multiplying it with 13.
Lets say you work at a startup that is growing insanely fast and you want to report financial metrics to investors, media etc. You can't use annual recurring revenue because 2026 is not over yet, and your company is so young it doesn't make sense to look back to last year. You can't use YoY because it would be some obscene figure (100000%) that definitely won't hold.
So the two best metrics are annualized recurring revenue (take last month * 12 or last 4 weeks * 13) and QoQ growth %.
There are two caveats:
- If the revenue is high quality (e.g. annual enterprise contracts, good NRR), then last 4 weeks * 13 is actually a conservative estimate as your company will likely continue to grow.
- But if the revenue is more volatile (e.g. consumption, token usage, bad NRR) then annualized recurring revenue can be used to hide worse performance because companies will juice revenue one month and report high "ARR"
I see but it's still predicted annual revenue. I can understand people not liking the word 'predicted' here because it is more grounding but that's what it is in the end? I guess i understand it now. thanks.
There’s a bunch of fuzzy metrics here, which is one reason I turned it back into a monthly number. The other issue (as you’ll see on the chart) is that Anthropic and openAI are recognising revenue in completely different ways.
Usually startups like to talk about Adjusted Annual Revenue in fundraising and other hype materials. There’s no regulation around this metric so whatever their investors are willing to accept is what they use. One way to measure it is to take the past 4 weeks revenue and multiply by 13.
Because that's 364 days.
52 weeks / 4 weeks = 13
13*4=52 weeks, mostly
Yeah it's weird huh? The "average" month contains 4.35 weeks.
(365/7)/12 = 4.3452…
28*13=364
52/4 = 13
Didn't Ben Evans previously shill for bitcoin, which is now omitted in the graphs for "disruptive technologies"?
This is a marketing Gish Gallop talk that pretends to invalidate counterarguments with a couple of fantasy graphs.
[flagged]
Yeah, beyond this mumbo-jumbo non-answer of yours, did you or did you not push crypto? Because if you did...it could kind of not speak for your analytical competence.
'these charts are fantasies' is a non-criticism from an anonymous moron. If there is an actual criticism of an actual point, make it.
Back when people were interested in Blockchains, I explained why people in tech were interested. I'm happy to explain that again now, if anyone cared. If someone thinks that's bad, they're a fool.
Blockchain as a technology was something a lot of people were interested in. I am curious about Bitcoin - the monopoly money of the real world. The questions was simple: did you, or did you not, promote Bitcoin? Anyone who would promote a fake, un-regulated "currency" would not be someone whose opinion I care about.
[dead]
Unflagged as I don't feel this comment deserves to be flagged. The call for reposting with a real name is unnecessary though - if an internet comment is incorrect or overstates the case, just reply to correct it or ignore it.
not to be too pedantic but sourced doesn't usually mean accurate. sourced can very well be fantasy. it will be a 'sourced fantasy' in that case or hallucination if you used a LLM.
Sure. So which chart does our anonymous coward think is a fantasy, and why?
Hah wow, what a way to confirm what he posted.
Why do you need this persons name?
The implication is that it's a bot saying this, not a person.
No it isn't. The implication is that pro-AI people can take revenge. He knows he is secure with his opinions. He even paraphrases Andreesen's title "software will eat the world". He has repeatedly appeared at a16z.
It is very secure to be pro-AI while the rest has to resort to unregistered typewriters like in the Soviet Union.
[flagged]
You are absolutely not a "random person on the internet", you are a very public figure with the wealth and influence available to you that could allow you to take revenge on someone who says something you don't like in real, life-altering ways. Now, I'm not saying that you would do that, but it is a shocking lack of self-awareness to think of yourself as a random person on the internet.
This is hilarious. What am I going to do? Hire a hitman? if you’re going be a dick on the Internet, you should be a dick using your real name. I don’t think that’s a threat.
Doesn't seem like a bot, and even if it were, the critique is germane. Calling for a name is a little threatening.
Looking at a 80 slide deck and saying that the charts are 'fantasies' is not a germane criticism at all. it's handwaving.
One of the graphs has two series: net revenue for one company, gross revenue for another. Absolutely ridiculous.
And that's just one example. You also haven't adjusted for inflation in your graphs that span multiple decades. Not to mention that the graphs themselves are not related to what you're discussing most of the time. You're just pointing at random historical developments and seemingly claiming they imply something for AI. They don't.
Also you don't name your sources. You just say "Companies" for most of them. Or a single name. Ridiculous. Those are not sources. You should identify the documents.
This is incredibly low quality work. A college freshman would do better.
All of these points are simply wrong.
I charted the revenue reported by Anthropic and OpenAI as gross and net because those are the numbers they disclose. Anthropic does not report net revenue nor give us any way to calculate that, and the same in reverse for OpenAI. It would be great if we had GAAP revenue, but we don't. This is what we have, and it still tells an important story. What are we supposed to do - just not show the data?
All of the charts indicate whether they are adjusted for inflation. There is no rigid convention on this, but the general practice is that you don't adjust up to maybe 20 years and generally do convert for more than that, unless inflation itself is under discussion, which it is not here. Meanwhile, the text on each slide explains the purpose of the comparison. None of them are random.
Every chart is correctly sourced exactly as you will see it in any other piece of financial or industry research. It is not industry practice to cite specific documents or provide footnotes.
You've clearly just never seen any industry or financial analysis before, and are unfamiliar with basic and universal conventions that run back decades. That's fine. Being rude about it is not.
It is not industry practice to actually give the sources then? Just list vague names in a short "Sources:Companies" note? I now see the inflation indication as well. Which is not on the y-axis label, but in the "Sources:"-note. I will concede that. But what the hell my guy, why is it down there.
If "financial analysis" is less rigid in sourcing requirements than grade school, what are you guys even doing. If you have the source, it's not very difficult to make a bibliography (or even write the year/publication with the author/publisher), and not doing so only serves to hinder the reader. If this is industry standard it means your entire industry is terrible at sourcing, does not want the reader to verify claims, or both.
And I've also definitely seen financial slide decks with actual sources cited. So I'm inclined to hope there are people in your industry who actually respect the reader's time.
Here's some more picks (and some more reasons you should list your sources): Your graphs based on surveys don't report error margins. You never list what a 100% is very precisely (what's the sample/population). In one graph, you don't label the y-axis at all (except for a 0 at the bottom)!
And finally, I was rude, yes. But I was only matching your energy. And I did show constraint there. I didn't make a veiled threat, did I?
You are surprised to discover how an entire industry you know nothing about does things. You conclude that everyone in that industry must be an idiot doing bad work.
This says far more about you than it does about me.
Yes, it says I have bare-minimum standards for how to cite references.
I never said I thought financial analysts are idiots. That's not a conclusion I made.
I gave a very reasonable reason: they want it to be more difficult to replicate the analysis. That makes sense if your goal is to make sure your work is profitable, rather than quality reporting and knowledge dissemination.
That would imply the people who set the standards (not everyone) is cynical and/or greedy. Not an idiot.
And still, presumably you are still allowed to add actual sources to financial analysis? You still failed the regular standard of good reporting on this. Your graphs also fail just regular data analysis sanity checks. There is, e.g., no thought given to whether it's even valid to compare data from two sources in one graph. You just do it.
It is an indisputable fact that you spent years shilling crypto. Why even deny that or threaten(!) someone pointing it out? It was/is a huge, verifiable chunk of your public output.
No, it's a really stupid lie. You can look at all my essays and presentations online.
I've spent some time discussing why people are interested in blockchains as software platforms, and what would be good and bad arguments around that. But I've never suggested anyone buy a token - indeed, I was pretty vocal in pointing to speculative bubbles and silly ideas, like NFTs.
Here you go:
https://en.cryptonomist.ch/2018/11/01/benedict-evans-cryptoc...
Nope. This is the exact quote.
Crypto today has a lot in common with both the internet in 1993 and the internet in 1999. Huge potential with few of the use cases invented yet, combined with froth, scams and delusion. This makes it easier to dismiss (“useless AND a scam!”).
But dismissing crypto as a useless scam is much like looking at Usenet, Cuecat and Boo .com and dismissing the internet. It mistakes applications for the enabling layer.
Looking at crypto and only seeing the scams is like looking at the internet in 1999 and only seeing the bubble.
Looking at crypto and seeing no use cases is like looking at the internet in 1993, when the web was 3% of traffic
Another parallel: 1993 - people complaining the term should be internets, not internet 2018 - people complaining 'that's not what crypto means'
You could argue I was wrong and blockchain's potential never turned into anything much. It actually has become a huge deal as plumbing in the finance industry, but not much else. But so what? This was an interesting tech that hasn't really worked out. Welcome to the tech industry.
I dunno... but those self-quotes sound a lot like at least implying that "missing out on crypto" would be like missing out on early days of Internet. You also seem to try and retrofit "blockchain" instead of "crypto", but these are two different things - blockchain is a technology, crypto is a form of play-money based on that technology, just like it's older brother, the monopoly-money, is based on the technology of paper (but not much else).
No, I was using crypto and blockchain as interchangeable terms. I'm aware some people don't, but that's always how any person the space I've spoken to uses them
Indeed you did, but that does not make it right. They are not interchangeable. Now again, did you, or did you not promote Bitcoin?
No, you have a different opinion about terminology, which ironically is one of the things I noted as pointless in the thread I posted.
No, answer my question - it's not that hard!
Nope what? You just confirmed the OP’s point. Glad we’re all in agreement.
I don't think it's fair to consider writing an analysis (even a favorable one) about a topic as _shilling_ for it. It's not like he was pushing shitcoins or minting NFTs.
I have always been (and remain!) bearish on crypto but it absolutely was something that couldn't be ignored a few years ago. Even if you came to the conclusion that it was bunk, there was significant enough fervor that any technologist needed to reckon with their position on it.
For example, lots of engineers proclaimed very loudly that document databases would replace the RDBMs, or that GraphQL was the future of APIs. They were wrong, as it turned out, but only with the wisdom of hindsight.
He actually was pushing NFTs but has since deleted his tweet: https://news.ycombinator.com/item?id=26434769
No, I didn’t. I made my account private and stopped using twitter. And the actual tweet says the opposite - that NFTs would need to develop some kind of cultural grounding for them to become a investment, which they didn’t have at the time and never got, and without that this would just be a speculative bubble, which is exactly what happened.
I made it very clear that I thought NFTs were a speculative bubble. I never suggested anyone should buy any crypto-related instrument. The idea that I was ‘shilling for crypto’ is something you would only say if you’re an idiot, as the OP and a few others on this thread clearly are.
[flagged]
That quote is actually from "Scott Rackey" and not the OP: https://web.archive.org/web/20220603095222/https://twitter.c...
No, I've never said that, and never thought that. The actual text in that tweet is: " The blockchain can't lie, but you can lie to the blockchain"
You don't know how to use Search, and this is beyond anything worth engaging with.
It's literally right there on the front page. People should go look for themselves.
It will literally eat the world. Just like we crowded out wild animals in a few reserved areas, so will AI data centers crowd us out.
To quite Ilya Sutskever:
> I think it’s pretty likely the entire surface of the earth will be covered with solar panels and data centers.
Or we could not do that...
Technology is meant to serve us not drive us into a hellscape lol
What's wrong with that? There are now materials that allow you to have solar panels on a window (so they are not opaque anymore), and we can put data centers under our feet.
Did you read the original comment?
> AI data centers crowd us out.
> the entire surface of the earth will be covered with solar panels and data centers.
Yes, so our buildings' windows will be solar panels, what's wrong with that? And all our floors can be data centers, what's wrong with that?
> the entire surface of the earth will be covered with solar panels and data centers.
You may want to lay off the LLM and practice your reading comprehension.
Ok I guess my mind didn't think "entire surface of the earth" would really mean entire surface of the earth because that's just abhorrently stupid. I mean, in order to cover the entire surface of the earth with datacenters, we would first need to build stuff on the entire surface of the earth... Which would take... thousands of years? And I'm sure we would grow some trees on top of the data-centers along the way.
In the current system technology is meant to serve the shareholders. That might end up serving us, if you believe the standard narrative. But maybe the shareholders will just carve out a few nature reserves for themselves and just wait for the rest of us to die off.
Time for a new system then!
"we"? Are we all now billionaires who decide what gets built and who gets laid off?
None of "us" gets to decide this. Only the very wealthy get to decide.
There's a way for us to have a say in the matter. It's happened before.
Okay, what way? What happened before?
Don't be vague.
You’re cop posting
The french perfected it.
Stop acting dumb. That person is obviously talking about a revolution.
> AI data centers crowd us out.
Here's a test to know if this is/will be true: look for a situation where the "needs" of AI (e.g. land, electricity, etc) conflict with the needs of people (e.g. land to live on, grow food on, electricity to light our homes).
Find a place where the needs of AI conflict with people, and observe who wins out.
Does the entity that owns the datacenter say, "oh sorry! I guess we're using too much electricity. No worries! We'll stop doing that" ...or does it say, "lol too bad, all the electricity belongs to us!"
Does the entity wanting to build a datacenter say, "oh sorry! We thought you'd be okay with us using this land. But if you're not that's okay, we wont build here" ...or does it say, "lol too bad, we own the government and they're seizing the land under eminent domain!"
(both of these scenarios have happened, btw)
I’ll put this in the hilarious category of “man outside his field of expertise”
> To quite Ilya Sutskever:
Who made Ilya Sutskever, or any other LLM-bonehead the Grand Prophet of Humanity? Why the fuck is his opinion on that relevant? Of course he will shill for data centers.
when someone gives their opinion about AI, one typical retort is "are you an AI/LLM expert? we should let the experts talk"
The so-called experts are bought and paid for.
The issue is that field moves so fast, that yesterday's experts are not so experts today.
it is silently assumed that only the experts who agree with me are the true experts
"Chat is a terrible UX General use needs ‘apps’"
I'm old so my computer career has gone: punch cards => calculators => command-line => GUI => touch screen => voice => chat. Chat seems to be the best blend of expressiveness and utility, with a dose of magic thrown in.
The ratio of AI startups at YC surprised me... (slide 48). This is a clear trend.
tl;dr;
> "What happened the last time that everything changed?"
Honestly, I'm glad we hear more of the commoditization of AI, and I hope that the comparison of AI with water or electricity will become mainstream and that the states (as in nation states) will understand that sooner rather than later and act accordingly.
How the fuck can you put ai on par with water or even electricity?
May be cuz it needs water and electricity for its existence?
> Automation takes a lot of manual labour
That's a great quote.
https://xkcd.com/1205/
This is excellent all around.
thank you for sharing. pdf really great!
What in the AI slop is the Yogi Berra “AI predications” duplicate slide?
There are no duplicate slides.
Ben, I follow you and think you’re brilliant, but boy you can’t take feedback like ever. Duplicate slide is there clearly to add the question, but it has a glaring typo—- “predications” instead of “predictions”. A random internet stranger read to page 51, which is already a rare occurrence, and helped you find a typo that you can now edit. But sure, the answer to that is “your comment is wrong”.
Ah, that’s different. Your question confused me because you said they were duplicate slides and they are not, and your word ‘predication’ looked like a typo by you.
The slides say "predication" and not "prediction." Just a typo :) I think the AI slop comment was excessive, and I liked your deck, but there is indeed a typo you could fix!
I believe they're referring to the quote showing up on 50 and 51 but that appears to be intentional (transition adds question below)
[dead]