AI's Brutally Concentrated Economics: 3% of Investments Generate 60% of Returns

Over the past couple of weeks, talk about an AI bubble and its fearsome implications has gone from a whisper to a full-throated roar. Not that that has appeared to dim investors' appetite for the sector. 

But one of the nuances emerging is whether one person's popped bubble may be another's opportunity. Specifically, the issue is the degree to which AI returns may be concentrated in a very few companies; much like the post-dot-com era, but occurring faster and at bigger multiples. AI concentration is not new news: Big Tech has dominated from the get-go and much of the payout is driven by the possibility of startup acquisition, not organic growth. So talk of a bubble may be more dependent on when investors finally give up on their smaller bets than on whether AI will really dominate the economy as it becomes ingrained over the 30 or so years history tells us it takes for a technology to become so deeply embedded as to be deemed essential. JL

Steven Rosenbush reports in the Wall Street Journal:

AI companies are losing money at an epic pace. The economics of AI have turned sharply against them, at least for now, and for reasons that weren’t widely anticipated.  There are increasingly urgent concerns about massive capital spending, soaring valuations, high debt levels and the circular nature of AI firms pouring money into other AI firms. (But) demand for AI, measured in units of data processed, is soaring right now. The entire AI bet may turn on how far and fast it ramps - and returns in AI may be very highly concentrated, leaving many contenders out in the cold. “In venture capital, 6% of investments result in 60% of returns,” venture capitalist Vinod Khosla told me. “In AI, I think it will be half that percentage resulting in more than 60% of returns.”

AI companies are losing money at an epic pace, and the reasons go deeper than mere profligacy. The economics of artificial intelligence have turned sharply against them, at least for now, and for reasons that weren’t widely anticipated.

You can’t blame people who believe a massive AI investment bubble is at risk of bursting, triggering a crisis on the scale of the dot-com bust or worse. There are increasingly urgent concerns about massive capital spending, soaring valuations, high debt levels and the circular nature of AI firms pouring money into other AI firms.

And sure, a handful of players would probably escape a sector collapse and go on to change the world, just like the dot-com survivors did. But even the most likely eventual winners in AI are losing billions of dollars right now.

It’s hard to predict how this will play out in the financial markets, but here’s a clue. Keep an eye on demand for AI, measured in units of data processed. It’s soaring right now. The entire AI bet may turn on how far and fast it ramps from here. 

A key mistake occurred a year ago when AI leaders focused solely on AI cost curves for a unit of computing and took their eye off the number of units needed to get the work done, said Heath Terry, the global sector lead for technology and communications research at financial services firm Citi. AI computing costs had been declining around 90% every seven months, a dynamic akin to Moore’s Law for microchips, giving AI companies reason to believe their price-performance ratio will improve.

Then usage, which everybody expected to increase as the price fell, went absolutely parabolic.

“This time last year, we were processing 9.7 trillion tokens a month across our products and APIs [application programming interfaces],” Alphabet’s Google said in a May blog post, referring to units of AI usage. “Now, we’re processing over 480 trillion—50 times more.” 

Google was just getting warmed up. “Since then we have doubled that number, now processing over 980 trillion monthly tokens, a remarkable increase,” the tech giant said in July during its second-quarter earnings call. 

This month, Google said the figure had reached 1.3 quadrillion.

(News Corp, owner of The Wall Street Journal, has a commercial agreement to supply content on Google platforms.)

More reliable answers, at a price

The demand for tokens has been driven in part by the fight against the “hallucinations,” or incorrect or nonsensical outputs stated as if they are true, notoriously produced by generative AI models. Current cutting-edge models are designed to counter the problem often with the help of a kind of reinforcement learning in which they answer the same question multiple times internally before issuing a response. They also might query a so-called mixture of experts, meaning specialized models. That has helped AI produce better answers—and exacerbated AI’s resource needs. So the unit price of a token has fallen, yes, but the overall consumption of tokens has gone nuts.

That demand will only continue to soar. OpenAI’s new Sora short-form AI video app powered by the latest version of its Sora video generator is a hit. (News Corp has a content-licensing partnership with OpenAI.) The company is looking to ramp up efforts in AI commerce as well. And Anthropic’s new large language model, Claude Sonnet 4.5, can code for 30 hours straight. Such longer-thinking models are an emerging form of AI that will put the industry’s ballooning data-center capacity to use.

AI startups are losing tens of billions of dollars more than they would have partly because of this dynamic. Startups and venture-capital investors so far are underwriting such losses, much as Uber Technologies once subsidized ride prices for users to build a market, Terry said.

Better economics, maybe

Three forces are on track to come together and push the economics of AI into the black, in Terry’s view.

First, models will continue to improve. While the evolution of the transformer architecture underlying generative AI such as large language models may have slowed down, developers are creating new forms of hybrid models such as neurosymbolic AI. Some AI developers say they are closer to the point where AI systems are beginning to assist in their own improvement, a concept that Terry calls AI squared.

“Over the last few months, we’ve begun to see glimpses of our AI systems improving themselves,” Meta Platforms Chief Executive Mark Zuckerberg said in July on the social-media giant’s second-quarter earnings call.

All of that should make AI models more efficient, which could have a beneficial effect on the economics of AI.

Second, data centers under construction around the world will soon begin operations, increasing the supply of computing power and energy and driving down the cost of training and using AI. Tech company Microsoft last month said it is in the final stages of construction on a $3.3 billion data center in Wisconsin and announced plans for a second, $4 billion facility in the area.

And third, increasing demand for tokens could finally become an economic benefit as models improve and the supply of computing infrastructure expands.

 

Another massive increase in demand for AI is likely in the coming months or quarters, just as those other forces take hold, according to Terry. That’s because the most promising corporate AI trials are likely to move into broader deployment.

AI computing will increase “by a billion X,” technology company Nvidia CEO Jensen Huang said last month on the “BG²” podcast. “That’s the part that most people haven’t completely internalized,” Huang said. “…This is the industrial revolution.” 

Should all of this go exactly according to plan, the AI boom will keep booming. Citi forecasts that AI revenue will grow nearly 80% annually over the next five years, reaching $780 billion in 2030, compared with $43 billion this year.

But even then, returns in AI may be very highly concentrated, eventually leaving many contenders out in the cold.

“In venture capital, 6% of investments result in 60% of returns,” venture capitalist Vinod Khosla told me, alluding to a 2015 blog post from a16z that cited three decades worth of data from investor Horsley Bridge. “In AI, I think it will be half that percentage resulting in more than 60% of returns.”

Khosla and other investors are betting that a handful of winners in their portfolios will be enough to offset their losses. It’s a pattern long-established in tech.

One thing is clear already, though. There are no half-measures here. Going “halfsies” on model development or infrastructure build-out won’t be enough to turn around the brutal economics of this tech. It’s an all-or-nothing bet on a trillion-dollar scale, and investors are all in.