The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? A car when in neutral will rev up easily if the accelerator pedal is pushed even very slightly, because there's no load on the engine. But in gear the same engine will rev up much less when the accelerator is pushed the same amount. Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising?
I doubt it.
And what if the technology to locally run these systems without reliance on the cloud becomes commonplace, as it now is with open source models? The expensive part is in the training of these models more than the inference.
Of all the players, I'd argue Google certainly knows how to give away a product for free and still make money.
The local open source argument doesn't hold water for me -- why does anyone buy Windows, Dropbox, etc when there's free alternatives?
We're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade.
People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.
> The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services?
Absolutely, not only are most AI services free but even the paid portion is coming from executives mandating that their employees use AI services. It's a heavily distorted market.
Will the OpenRouter marketplace of M clouds X N models die if the investor money stops? I believe its a free and profitable service, offered completely pay as you go.
The thing that makes AI investment hard to reason about for individuals is that our expectations are mostly driven by a single person’s usage, just like many of the numbers reported in the article.
But the AI providers are betting, correctly in my opinion, that many companies will find uses for LLM’s which are in the trillions of tokens per day.
Think less of “a bunch of people want to get recipe ideas.”
Think more of “a pharma lab wants to explore all possible interactions for a particular drug” or “an airline wants its front-line customer service fully managed by LLM.”
It’s unusual that individuals and industry get access to basically similar tools at the same time, but we should think of tools like ChatGPT and similar as “foot in the door” products which create appetite and room to explore exponentially larger token use in industry.
Some of the utilization comparisons are interesting, but the article says 2 trillion was spent on laying fiber but that seems suspicious.
There's an enormous amount of unused, abandoned fiber. All sorts of fiber was run to last mile locations, across most cities in the US, and a shocking amount effectively got abandoned in the frenzy of mergers and acquisitions. 2 trillion seems like a reasonable estimate.
Giant telecoms bought big regional telecoms which came about from local telecoms merging and acquiring other local telecoms. A whole bunch of them were construction companies that rode the wave, put in resources to run dark fiber all over the place. Local energy companies and the like sometimes participated.
There were no standard ways of documenting runs, and it was beneficial to keep things relatively secret, since if you could provide fiber capabilities in a key region, but your competition was rolling out DSL and investing lots of money, you could pounce and make them waste resources, and so on. This led to enormous waste and fraud, and we're now on the outer edge of usability for most of the fiber that was laid - 29-30 years after it was run, most of it will never be used, or ever have been used.
The 90s and early 2000's were nuts.
For infrastructure, central planning and state-run systems make a lot of sense - this after all is how the USA's interstate highway system was built. The important caveat is that system components and necessary tools should be provided by the competitive private sector through transparent bidding processes - eg, you don't have state-run factories for making switches, fiber cable, road graders, steel rebar, etc. There are all kinds of debatable issues, eg should system maintenance be contracted out to specialized providers, or kept in-house, etc.
I so desperately wish it weren't abandoned. I hate that it's almost 2026 and I still can't get a fiber connection to my apartment in a dense part of San Diego. I've moved several times throughout the years and it has never been an option despite the fact that it always seems to be "in the neighborhood".
Stylistically, this smells like it was copy and pasted from straight out Deep Research. Substantively, I could use additional emphasis on the mismatch between expectations and reality with regards to telco debt-repayment schedule.
Yes or no conclusions aside (and despite its title, the article deserves better than that), the key point is I think this one: “But unlike telecoms, that overcapacity would likely get absorbed.”
Telecom (dark fiber) capacity got absorbed too. Eventually. After a ton of bankruptcies.
Don’t think looking at power consumption of b200s is a good measure of anything. Could well be an indication of higher density rather than hitting limits and cranking voltage to compensate
Yes, one of NVidia's selling points for the b200 is that performance per watt is better than before. High power consumption without controlling for performance means nothing.
Is there a way in which this is good for a segment of consumers? When the current gen of GPUs are too old, will the market be flooded with cheap GPUs that benefit researchers and hobbyists who therwis would not afford them?
GPUs age surprisingly gracefully. If a GPU isn't cutting edge, you just tie two or more of them together for a bit more power consumption to get more or less the same result as the next generation GPU.
if there's ever a glut in GPUs that formula might change but it sure hasn't happened yet. also, people deeply underestimate how long it would take a competing technology to displace them. It took GPUs nearly a decade and the fortunate occurrence of the AI boom to displace CPUs in the first place despite bountiful evidence in HPC that they were already a big deal.
Some of them will probably be starving, homeless, or bedridden by the time that happens but yes they can get cheap GPUs
Many researchers and hobbyists cannot even plug in a 10 KW 8 GPU DGX server.
That doesn't exactly bode well for the EV revolution, then, does it?
The average commute in the United States is about 24 miles a day round trip. That's about 10 kWH. That's enough to charge overnight on a 15A circuit.
wut?
Unlikely, for a few reasons:
* The GPUs in use in data centers typically aren’t built for consumer workloads, power systems, or enclosures.
* Data Centers often shred their hardware for security purposes, to ensure any residual data is definitively destroyed
* Tax incentives and corporate structures make it cheaper/more profitable to write-off the kit entirely via disposal than attempt to sell it after the fact or run it at a discount to recoup some costs
* The Hyperscalers will have use for the kit inside even if AI goes bust, especially the CPUs, memory, and storage for added capacity
That’s my read, anyway. They learned a lot from the telecoms crash and adjusted business models accordingly to protect themselves in the event of a bubble crash.
We will not benefit from this failure, but they will benefit regardless of its success.
Nice article; far from bullet-proof, but it brings up some interesting points. HN comments are vicious on the topic of AI non-bubbles.
No. AI data center, or any data center is designed with incorrect data structure resulting in over utilisation of computing resource.
This seems to be either LLM AI slop or a person working very hard to imitate LLM writing style:
The key dynamic: X were Y while A was merely B. While C needed to be built, there was enormous overbuilding that D ...
Why Forecasting Is Nearly Impossible
Here's where I think the comparison to telecoms becomes both interesting and concerning.
[lists exactly three difficulties with forecasting, the first two of which consist of exactly three bullet points]
...
What About a Short-Term Correction?
Could there still be a short-term crash? Absolutely.
Scenarios that could trigger a correction:
1. Agent adoption hits a wall ...
[continues to list exactly three "scenarios"]
The Key Difference From S:
Even if there's a correction, the underlying dynamics are different. E did F, then watched G. The result: H.
If we do I and only get J, that's not K - that's just L.
A correction might mean M, N, and O as P. But that's fundamentally different from Q while R. ...
The key insight people miss ...
If it's not AI slop, it's a human who doesn't know what they're talking about: "enormous strides were made on the optical transceivers, allowing the same fibre to carry 100,000x more traffic over the following decade. Just one example is WDM multiplexing..." when in fact wavelength division multiplexing multiplexing is the entirety of those enormous strides.
Although it constantly uses the "rule of three" and the "negative parallelisms" I've quoted above, it completely avoids most of the overused AI words (other than "key", which occurs six times in only 2257 words, all six times as adjectival puffery), and it substitutes single hyphens for em dashes even when em dashes were obviously meant (in 20 separate places—more often than even I use em dashes), so I think it's been run through a simple filter to conceal its origin.
Remember we have about 20 years of poorly written articles along with a few well written ones for the LLM to be trained on. I'm confident that attempting to tell LLM from human writing is a waste of time now that the year is almost over.
Other than that I'd rather choose a comprehensive article than a summary.
Attempting to tell LLM writing from human writing is usually a waste of time, but apparently not in this case.
I agree, and it feels like an allergy by now to that style specifically. This is doubly annoying because it ruins the reading experience and just makes me question myself constantly because you often can't be quite certain especially for shorter posts/comments.
On topic: It is always quite easy to be the cynical skeptic, but a better question in my view: Is the current AI boom closer to telecoms in 2000 or to video hosting in 2005? Because parallels are strong to both, and the outcomes vastly different (Cisco barely recovered by now compared to 1999 while youtube is printing money).
Yes
Hardware growth is slow and predictable, but one breakthrough algorithm completely undercuts any finance hypothesis premised on compute not flowing out of the cloud and back to the edges and into the phones.
This is a kind of risk that finance people are completely blind to. Open AI won't tell them because it keeps capital cheap. Startups that must take a chance on hardware capability remaining centralized won't even bother analyzing the possibility. With so many actors incentivized to not know or not bother asking the question, there's the biggest systematic risk.
The real whiplash will come from extrapolation. If an algorithm advance shows up promising to halve hardware requirements, finance heads will reason that we haven't hit the floor yet. A lot of capital will eventually re-deploy, but in the meantime, a great deal of it will slow down, stop, or reverse gears and get un-deployed.
AI had a kind of Jevons paradox approach to efficiency improvements, unfortunately - if you halve the compute requirements with an algorithmic advance, you can run a model twice as big.
> This is the opposite of what happened in telecoms. We're not seeing exponential efficiency gains that make existing infrastructure obsolete. Instead, we're seeing semiconductor physics hitting fundamental limits.
What about the possibility of improvements in training and inference algorithms? Or do we know we won't get any better than grad descent/hessians/etc ?
Holly cow, we've found an exception to Betteridge's Law of Headlines! Talk about burying the lede!
If you read the article, then this is not an exception to the law
The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services? A car when in neutral will rev up easily if the accelerator pedal is pushed even very slightly, because there's no load on the engine. But in gear the same engine will rev up much less when the accelerator is pushed the same amount. Will there be the same overutilisation occurring if users have to financially support the infrastructure, either through subscriptions or intrusive advertising?
I doubt it.
And what if the technology to locally run these systems without reliance on the cloud becomes commonplace, as it now is with open source models? The expensive part is in the training of these models more than the inference.
Of all the players, I'd argue Google certainly knows how to give away a product for free and still make money.
The local open source argument doesn't hold water for me -- why does anyone buy Windows, Dropbox, etc when there's free alternatives?
We're talking miraculous level of improvement for a SOA LLM to run on a phone without crushing battery life this decade.
People are missing the forest for the trees here. Being the go to consumer Gen AI is a trillion+ dollar business. How many 10s of billions you waste on building unnecessary data centers is a rounding error. The important number is your odds of becoming that default provider in the minds of consumers.
> The article claims that AI services are currently over-utilised. Well isn't that because customers are being undercharged for services?
Absolutely, not only are most AI services free but even the paid portion is coming from executives mandating that their employees use AI services. It's a heavily distorted market.
Will the OpenRouter marketplace of M clouds X N models die if the investor money stops? I believe its a free and profitable service, offered completely pay as you go.
The thing that makes AI investment hard to reason about for individuals is that our expectations are mostly driven by a single person’s usage, just like many of the numbers reported in the article.
But the AI providers are betting, correctly in my opinion, that many companies will find uses for LLM’s which are in the trillions of tokens per day.
Think less of “a bunch of people want to get recipe ideas.”
Think more of “a pharma lab wants to explore all possible interactions for a particular drug” or “an airline wants its front-line customer service fully managed by LLM.”
It’s unusual that individuals and industry get access to basically similar tools at the same time, but we should think of tools like ChatGPT and similar as “foot in the door” products which create appetite and room to explore exponentially larger token use in industry.
Some of the utilization comparisons are interesting, but the article says 2 trillion was spent on laying fiber but that seems suspicious.
There's an enormous amount of unused, abandoned fiber. All sorts of fiber was run to last mile locations, across most cities in the US, and a shocking amount effectively got abandoned in the frenzy of mergers and acquisitions. 2 trillion seems like a reasonable estimate.
Giant telecoms bought big regional telecoms which came about from local telecoms merging and acquiring other local telecoms. A whole bunch of them were construction companies that rode the wave, put in resources to run dark fiber all over the place. Local energy companies and the like sometimes participated.
There were no standard ways of documenting runs, and it was beneficial to keep things relatively secret, since if you could provide fiber capabilities in a key region, but your competition was rolling out DSL and investing lots of money, you could pounce and make them waste resources, and so on. This led to enormous waste and fraud, and we're now on the outer edge of usability for most of the fiber that was laid - 29-30 years after it was run, most of it will never be used, or ever have been used.
The 90s and early 2000's were nuts.
For infrastructure, central planning and state-run systems make a lot of sense - this after all is how the USA's interstate highway system was built. The important caveat is that system components and necessary tools should be provided by the competitive private sector through transparent bidding processes - eg, you don't have state-run factories for making switches, fiber cable, road graders, steel rebar, etc. There are all kinds of debatable issues, eg should system maintenance be contracted out to specialized providers, or kept in-house, etc.
I so desperately wish it weren't abandoned. I hate that it's almost 2026 and I still can't get a fiber connection to my apartment in a dense part of San Diego. I've moved several times throughout the years and it has never been an option despite the fact that it always seems to be "in the neighborhood".
Stylistically, this smells like it was copy and pasted from straight out Deep Research. Substantively, I could use additional emphasis on the mismatch between expectations and reality with regards to telco debt-repayment schedule.
Yes or no conclusions aside (and despite its title, the article deserves better than that), the key point is I think this one: “But unlike telecoms, that overcapacity would likely get absorbed.”
Telecom (dark fiber) capacity got absorbed too. Eventually. After a ton of bankruptcies.
Don’t think looking at power consumption of b200s is a good measure of anything. Could well be an indication of higher density rather than hitting limits and cranking voltage to compensate
Yes, one of NVidia's selling points for the b200 is that performance per watt is better than before. High power consumption without controlling for performance means nothing.
Is there a way in which this is good for a segment of consumers? When the current gen of GPUs are too old, will the market be flooded with cheap GPUs that benefit researchers and hobbyists who therwis would not afford them?
GPUs age surprisingly gracefully. If a GPU isn't cutting edge, you just tie two or more of them together for a bit more power consumption to get more or less the same result as the next generation GPU.
if there's ever a glut in GPUs that formula might change but it sure hasn't happened yet. also, people deeply underestimate how long it would take a competing technology to displace them. It took GPUs nearly a decade and the fortunate occurrence of the AI boom to displace CPUs in the first place despite bountiful evidence in HPC that they were already a big deal.
Some of them will probably be starving, homeless, or bedridden by the time that happens but yes they can get cheap GPUs
Many researchers and hobbyists cannot even plug in a 10 KW 8 GPU DGX server.
That doesn't exactly bode well for the EV revolution, then, does it?
The average commute in the United States is about 24 miles a day round trip. That's about 10 kWH. That's enough to charge overnight on a 15A circuit.
wut?
Unlikely, for a few reasons:
* The GPUs in use in data centers typically aren’t built for consumer workloads, power systems, or enclosures.
* Data Centers often shred their hardware for security purposes, to ensure any residual data is definitively destroyed
* Tax incentives and corporate structures make it cheaper/more profitable to write-off the kit entirely via disposal than attempt to sell it after the fact or run it at a discount to recoup some costs
* The Hyperscalers will have use for the kit inside even if AI goes bust, especially the CPUs, memory, and storage for added capacity
That’s my read, anyway. They learned a lot from the telecoms crash and adjusted business models accordingly to protect themselves in the event of a bubble crash.
We will not benefit from this failure, but they will benefit regardless of its success.
Nice article; far from bullet-proof, but it brings up some interesting points. HN comments are vicious on the topic of AI non-bubbles.
No. AI data center, or any data center is designed with incorrect data structure resulting in over utilisation of computing resource.
This seems to be either LLM AI slop or a person working very hard to imitate LLM writing style:
The key dynamic: X were Y while A was merely B. While C needed to be built, there was enormous overbuilding that D ...
Why Forecasting Is Nearly Impossible
Here's where I think the comparison to telecoms becomes both interesting and concerning.
[lists exactly three difficulties with forecasting, the first two of which consist of exactly three bullet points]
...
What About a Short-Term Correction?
Could there still be a short-term crash? Absolutely.
Scenarios that could trigger a correction:
1. Agent adoption hits a wall ...
[continues to list exactly three "scenarios"]
The Key Difference From S:
Even if there's a correction, the underlying dynamics are different. E did F, then watched G. The result: H.
If we do I and only get J, that's not K - that's just L.
A correction might mean M, N, and O as P. But that's fundamentally different from Q while R. ...
The key insight people miss ...
If it's not AI slop, it's a human who doesn't know what they're talking about: "enormous strides were made on the optical transceivers, allowing the same fibre to carry 100,000x more traffic over the following decade. Just one example is WDM multiplexing..." when in fact wavelength division multiplexing multiplexing is the entirety of those enormous strides.
Although it constantly uses the "rule of three" and the "negative parallelisms" I've quoted above, it completely avoids most of the overused AI words (other than "key", which occurs six times in only 2257 words, all six times as adjectival puffery), and it substitutes single hyphens for em dashes even when em dashes were obviously meant (in 20 separate places—more often than even I use em dashes), so I think it's been run through a simple filter to conceal its origin.
Remember we have about 20 years of poorly written articles along with a few well written ones for the LLM to be trained on. I'm confident that attempting to tell LLM from human writing is a waste of time now that the year is almost over.
Other than that I'd rather choose a comprehensive article than a summary.
Attempting to tell LLM writing from human writing is usually a waste of time, but apparently not in this case.
I agree, and it feels like an allergy by now to that style specifically. This is doubly annoying because it ruins the reading experience and just makes me question myself constantly because you often can't be quite certain especially for shorter posts/comments.
On topic: It is always quite easy to be the cynical skeptic, but a better question in my view: Is the current AI boom closer to telecoms in 2000 or to video hosting in 2005? Because parallels are strong to both, and the outcomes vastly different (Cisco barely recovered by now compared to 1999 while youtube is printing money).
Yes
Hardware growth is slow and predictable, but one breakthrough algorithm completely undercuts any finance hypothesis premised on compute not flowing out of the cloud and back to the edges and into the phones.
This is a kind of risk that finance people are completely blind to. Open AI won't tell them because it keeps capital cheap. Startups that must take a chance on hardware capability remaining centralized won't even bother analyzing the possibility. With so many actors incentivized to not know or not bother asking the question, there's the biggest systematic risk.
The real whiplash will come from extrapolation. If an algorithm advance shows up promising to halve hardware requirements, finance heads will reason that we haven't hit the floor yet. A lot of capital will eventually re-deploy, but in the meantime, a great deal of it will slow down, stop, or reverse gears and get un-deployed.
AI had a kind of Jevons paradox approach to efficiency improvements, unfortunately - if you halve the compute requirements with an algorithmic advance, you can run a model twice as big.
> This is the opposite of what happened in telecoms. We're not seeing exponential efficiency gains that make existing infrastructure obsolete. Instead, we're seeing semiconductor physics hitting fundamental limits.
What about the possibility of improvements in training and inference algorithms? Or do we know we won't get any better than grad descent/hessians/etc ?
Holly cow, we've found an exception to Betteridge's Law of Headlines! Talk about burying the lede!
If you read the article, then this is not an exception to the law