Yet another Hacker News

IBM CEO says open weight models are now irresistibly good and cheap on-prem

72 points Betelbuddy 2 hours ago 33 comments

jains99 59 minutes ago | parent

But hardware to run them is conveniently something IBM would be happy to sell you.

anukin 51 minutes ago | parent

Not till Jensen buys his next leather jacket.

jacquesm 32 minutes ago | parent

If IBM would ship a piece of affordable hardware that can run large LLMs efficiently I'd be lining up.

solyin 29 minutes ago | parent

"Affordable" and "IBM" don't belong in the same sentence together, but they're happy to sell you their inference-capable hardware if you've got the wallet for it.

jacquesm 25 minutes ago | parent

I don't agree with that. They made the PC revolution possible by making hardware that was at least affordable enough to be bought by a large number of businesses and individuals. It also set the stage for the clone market. I'm pretty sure that if IBM really set their mind to it that they could do this.

That doesn't detract from your overall point that they tend to make overpriced stuff, but I don't think they have to do that. Besides, what's overpriced? NVidia's GB300 workstation sells for 95K euros, a single RTX 6000 is now 20K. Surely even IBM can compete at that level.

quailfarmer 21 minutes ago | parent

They haven’t been _that_ IBM for decades

jacquesm 12 minutes ago | parent

I know. But: they have gobs of money, they still have some talent, they still have a hardware department that I'm pretty sure would jump at a chance to give NV a run for their money. This would require a massive long term investment and a commitment that I'm not sure they would have the stomach for.

dansquizsoft 18 minutes ago | parent

> They made the PC revolution possible

Yeah they were only half a decade late to the PC revolution back in the day...

rnd0 17 minutes ago | parent

That was completely accidental and they themselves tried to close the barn door with the PS/2 line in 1987 but at that point it was too late.

jacquesm 6 minutes ago | parent

True, even so: They still make some pretty impressive hardware. They have access to fab capacity, they know how to make chips. They have the capital. If they really wanted they could do it.

numpad0 18 minutes ago | parent

Yes but unironically? IBM should be what NVIDIA would be competing with their NVL72 and DGX products. It doesn't make sense that IBM is just watching NVIDIA take its lunch.

CookieCrisp 50 minutes ago | parent

Good enough for the old world, but I'm not so sure it's good enough for what's coming.

tomrod 48 minutes ago | parent

What do you think is coming that isn't already in the old world?

Cerebras with Claude 8?

LoganDark 26 minutes ago | parent

OpenAI most likely pushed Cerebras into a contractual agreement not to do business with Anthropic through 2030. ("Most likely" because the names of the companies Cerebras is contractually obligated not to sell to are redacted in public filings)

collinmcnulty 45 minutes ago | parent

Can we please stop giving weight to CEOs talking their book? Like of course IBM wants you to think this, it would be great for them if it’s true. And maybe it is true! But IBM’s CEO saying it’s true tells me just as much as if a frontier lab said it’s not: nothing.

MattDamonSpace 39 minutes ago | parent

It’s a conversation point off which a discussion can be had

gjvc 31 minutes ago | parent

the only credible "conversation" is "well he would say that, wouldn't he?"

nvme0n1p1 39 minutes ago | parent

"CEO Said A Thing!" Journalism

https://karlbode.com/ceo-said-a-thing-journalism/

epsteingpt 35 minutes ago | parent

But you know - it would be funny if somehow Open Weights are the thing that completely save IBM

SoftTalker 28 minutes ago | parent

Linux already served as a life-preserver for them also.

gjvc 30 minutes ago | parent

"we are delighted..." said a spokesman

trollbridge 41 minutes ago | parent

  While he thinks most businesses can build useful AI infrastructure in just two or three racks, he feels many will struggle to get it running.

  “The question becomes: how do you do that?” he said. “The question that the enterprise struggles with is: ‘Where are my kernel engineers? Where are the people who worry about the underlying network? Who can make sure that the model is clustered on GPUs that are close to each other in network proximity? Who can make sure that you’re doing the right caching of results so you get an incredible response?’”

Has he heard of “agentic AI”? You simply fire up Claude or ChatGPT and ask it how to do all of the above.

Buttons840 28 minutes ago | parent

That's based on the idea that humans can learn and adapt and solve problems, and that's not what companies want to think about when they're seeking a profitable transaction such as hiring.

LoganDark 27 minutes ago | parent

Do not do that if you can't prompt it correctly. It is a potential disaster waiting to happen. LLMs are likely to pull things off the shelf without real defense in depth and that's a problem. Maybe slightly less if it's only in an internal network, but still.

jacquesm 34 minutes ago | parent

It's absolutely true. I have three different setups here, a rig with 8 gpus that I use in either one of two modes (all 8, very heavy model or just one much lighter) and two DGX sparks. I haven't used a paid model in months. Sure, they use a few more tokens and the results may be slightly less good (but this doesn't really matter because I'm still in the loop so it is easy to correct). But the fact that I'm not sharing my work with the paid providers is much more to my liking than having big brother looking over my shoulder at every step and occasionally giving me the 'I can't let you do that, Jacques' treatment.

otterley 29 minutes ago | parent

How much did you shell out for this setup? What’s the TTFT and the output tokens/sec rate? How much electricity does it consume?

jacquesm 23 minutes ago | parent

Working on a blog post to document it all, but that will take a few more days. The short version: the big rig is probably $25K, ttft is dependent on how much prefill you have but typically a couple of seconds (2K prefill/sec in one setup, about 500 in the other), tgs 100 to 150 or so depending on what it is doing and how well the speculative decoder works with the current context. Electricity is pretty bad, full tilt it's about 2500W or so but I've power limited it to 1500 to keep it civil.

dansquizsoft 20 minutes ago | parent

> Sure, they use a few more tokens and the results may be slightly less good (but this doesn't really matter because I'm still in the loop so it is easy to correct)

I can't wait to use models that are 'slightly less good' when I am at work instead of something like Fable that lets me focus on the problem at hand as much as possible rather than also spending additional effort correcting for the tool I am using...

sp527 7 minutes ago | parent

You need to reread what was written. No one is proposing to take away your right to leak proprietary edge to the rapacious frontier labs that need to eat most of the economy in order to survive.

djelu 22 minutes ago | parent

Considering the disclosed numbers for DeepInfer and the usefulness/popularity of DeepSeek 4.1 Flash, I expect it is inevitable that more companies are going to consider the self-hosting option seriously, and perhaps play with the idea of offering inference as a vendor.

matt3210 18 minutes ago | parent

RIP frontier AI, the top customers can easily self host