YHK - yet another hacker news clone

Fiber lasts decades, GPUs last three years: the AI boom isn't the dot-com boom [video]

35 points frag 3 hours ago 39 comments

smashspectacle 2 hours ago | parent

Ed Zitron is going to become the Simpsons of ai critiques...

pllbnk 59 minutes ago | parent

Lots of respect for him for staying unwaveringly principled while knowing when and how to adjust his expectations and predictions. He is really well informed and it's something else to watch him speaking live on the podcasts and be so specific with numbers. He really knows his stuff.

saagarjha 56 minutes ago | parent

He really does not.

sevenzero 53 minutes ago | parent

Good point, very well structured argument here

ericd 46 minutes ago | parent

Go read "Some Zitron predictions" here: https://danluu.com/zitron/

His track record of prediction on this topic is stunningly bad, and he's made a job out of being counter to the AI hype, so he seems likely unable to honestly update his beliefs to adapt to changing evidence without jeopardizing the attention people pay to him.

pllbnk 30 minutes ago | parent

Predicting global events is next to impossible. Don't look at the predictions, as his prediction game has slowed and he has recently admitted that he was wrong on his predictions. Look at his exposing of financial shenanigans which let the bubble grow.

ericd 6 minutes ago | parent

Sure, which is part of why most people try not to do it publicly with much confidence, lest they look like they're not worth listening to on the topic. Like, why are we listening to his opinions on this when he so clearly isn't an expert?

And I think a big part of why you shouldn't listen to him is that he's said repeatedly that he doesn't really use generative AI outside of occasional experiments - he hasn't seen its potential enough to bother using it himself beyond some initial tests. Which is fine, but I think it also means that he likely doesn't have a good intuition of the likely future demand, which is pretty key in modeling whether debt loads are sustainable or not. Because investing is fundamentally about trying to predict the future and putting your money into the things that will likely be successful there. And he's demonstrably been terrible at predicting the future of this stuff.

jdietrich 2 hours ago | parent

Siri, what's the spot price for an A100?

orwin 2 hours ago | parent

I don't want to argue here, but once models are trained, using them is quite cheap. Also in the last few months most improvements I've seen are in the smaller sized models, which might make using models even cheaper.

nisegami 2 hours ago | parent

There's this weird tension where small models are getting a lot better, but the gap between the frontier and "what I can run on my gaming GPU" at home isn't necessarily getting smaller. That said, Qwen-3.8-27B-Q3 runs on my 9700 XT and performs well enough for tool calling in agents, which is a critical floor in local model performance imo.

TonyStr 1 hour ago | parent

Do you use qwen for any coding? Do you make a frontier model orchestrate it? Are you running with llama.cpp? Sorry for the barrage of questions, but I've been interesting in local AI for a while, yet I've struggled to set up anything usable.

esseph 1 hour ago | parent

Just go download and install lemonade server. Open web ui for front end. There are mobile apps to connect to it.

https://lemonade-server.ai/

Driver4732 2 hours ago | parent

If the models are cheap to run, while good for customers, isn't that a problem for hyperscalers and all the investment poured into the AI data center build out?

BryanBigs 1 hour ago | parent

All of this spending only has a chance of paying off if video generation takes off.

skeezyboi 47 minutes ago | parent

its a false equivalence.... the dotcom boom wasnt just a fibre rollout, it was all the internet company startups that were overpriced and crashed. no doubt some of the hyperscalars will get busted out, but the models and LLMs are here to stay. just like dot com websites didnt disappear after the dot com crash, neither will AI transformer models

KevinMS 1 hour ago | parent

> using them is quite cheap.

until the other guys comes out with a slightly better model, then using yours wont be cheap enough.

deadlast2 2 hours ago | parent

If you use opencode for your AI work or becomes affordable. To the point where you would reasonably build applications with intelligence the LLM version. I think in the near future the intelligence will be free and there is no business in free. I know it is a big statement but if you look around there is already a large amount of free models of gradually increasing strength. For most applications you don't need a gpt6

mono442 1 hour ago | parent

Another thing to pay attention to is the margins of hardware companies like nvidia or micron. Hyperscalers are spending huge amount of money but the money mostly goes to the profits of hardware companies.

DennisP 1 hour ago | parent

Bad for hyperscalers. Good for Nvidia.

ndom91 1 hour ago | parent

Shameless self promotion: I put together an AI infra built-out financials data-vis project/page recently. It includes graphs and tracks updates to stuff like GPU depreciation schedules like this.

https://artificialliabilities.com

LilBytes 48 minutes ago | parent

I love the presentation of this. Phenomenal work, thank you.

Bears are gonna eat.

gavmor 8 minutes ago | parent

Oops, I read it as bullish. 98.6% utilization? Seems like demand for more capacity.

saidinesh5 55 minutes ago | parent

GPUs might be 3 years but the rest of the hardware: processor cores, ram, storage can power other cloud applications for a decade or more no?

ericd 47 minutes ago | parent

Except the GPUs last a lot longer than 3 years - A100s are 6 years old at this point and still in heavy use.

adrian_b 34 minutes ago | parent

Some GPUs may easily last more than 10 years of light use, but I am not aware of public data about the reliability of datacenter GPus, which would give the percentage of A100s that die during the first 3 years of 24/7 use and during the first 6 years of 24/7 use.

I doubt that many of the 6-year old A100s have been in use 24/7.

whynotminot 39 minutes ago | parent

I recommend looking at any data regarding A100 rents (6 year old GPU) and then save 50 minutes of your time.

molyss 30 minutes ago | parent

What are we looking for? What’s your time-saving insight?

paxy 5 minutes ago | parent

The fact that the claim "GPUs last three years" is idiotic.

nyeah 4 minutes ago | parent

[delayed]

samuelknight 16 minutes ago | parent

Same with the consumer variants. RTX 3090 launched at $1499 and used ones on ebay are over $1500. GPUs large enough and new enough to do inference are not losing value.

Volta generation prices are collapsed, so GPU depreciation is currently around 8 or 9 years

ActionHank 10 minutes ago | parent

Thats the weird thing, scarcity drives value up so long as there is scarcity.

Once there is enough, or even a glut, it's over.

Good4boothee 37 minutes ago | parent

There are multiple YT channels proclaiming Russia is finally collapsing/losing war with Ukraine, huge "It is happening" in thumbnail; few of them put out a variation of such video every week, not even bothering to excuse their previous, wrong prediction. And any reporting related to AI bubble is exactly as terrible. I might as well just check Bloomberg/FT/QZ, news there will be devoid of technical details and still more useful than those endless armchair-experts' speculations.

anu7df 28 minutes ago | parent

GPUs may last 6 yrs or more. Fiber vs GPUs is however not the right comparison. Fiber vs Open models are. Open models have mostly caught up with frontier models. Now, with a reasonable harness and tools they can easily be useful for vast majority of users for a decade or more.

cmiles8 27 minutes ago | parent

There is a use for older GPUs, but the chips do have reliability issues. Turns out running chips hard 24/7 causes them to burn out. So it’s less will they be “useful” a while from now but more will they still work.

Folks are leaning it’s like these LED light bulbs that say they’ll last 15 years but after 2 years they start flaking out at a high rate.

Separately, the broader point is still directionally correct. We’re still using fiber from the .com network infra implosion today and will for a long time. The deprecation cycle is incredibly long. The increasing reports of GPUs sitting on shelves because there aren’t data center to power them is very concerning considering all the high interest debt being used to buy what is ultimately a rapidly depreciating asset. Financially, that’s a disaster waiting to happen.

booty 13 minutes ago | parent

    Turns out running chips hard 24/7 causes them to burn out
I don't have direct experience running CPUs or GPUs 24/7, but when I was looking into acquiring used GPUs I looked into this and it seemed like "burning out GPUs" was almost entirely related to the thermal paste cooking off and other heat-related issues. The tribal wisdom seemed to be that GPUs could run pretty much indefinitely if you kept their thermals under control and reapplied the thermal paste every N months.

Hopefully somebody with direct experience can correct me if I'm wrong...

ActionHank 8 minutes ago | parent

No, take a look into people who bought used GPUs that were from crypto farms. It's not just thermal paste, the chips become unreliable because of degradation of the circuits etched on the chips themselves.

robochat 5 minutes ago | parent

Which is almost exactly the same story that applied to the flaky LEDs - bad thermal designs leads to the LEDs or their driver circuitry breaking.

nyeah 6 minutes ago | parent

[delayed]