YHK - yet another hacker news clone

If Apple, in 2030 has a compute product that a worker can BYO(X) to work that has 2TB RAM, with an encrypted personalized context that can float between agents, then they will be in a very interesting place.

But the family shouldn’t be limited by that. We’re not going to see M6 Max hardware, because the M7 line is being fast-tracked and pulled forward. And I wouldn’t be at all surprised to see M7 capable of 1-2TB or more.

What new bike should I get? How should I rephrase this email? What to do about my persistent headaches? Recipes based on these ingredients?

It'll become easier and more affordable to distill/train/optimize smaller local models to meet your specific work needs. Multi-agent efforts will orchestrate the balance between using on-prem hardware for most tasks, and then defer to cloud models for more demanding/edge-case tasks. Especially if an organization can manage what data is shared with which model.

I also expect cloud providers to eventually start charging different rates for on-peak and off-peak usage, which would incentivize running what you can with the local compute you have otherwise sitting idle across dozens to hundreds of machines in your fleet.

I could be wrong and all your future hardware purchases will be glorified thin clients, with less local processing capabilities than today's laptops, servers, and towers.

In the mainframe's heyday, it was difficult to grasp a world in which compute became so dense and so cheap that it made sense for each individual to have all processing and memory on their desk, talk less of on their lap and in their hand.

Of course this time could be different. But, when we look at previous relentless progress in compute and efficiency, then also couple that with the breakthroughs AI itself unlocks, it's hard for me to imagine that centralized AI will meet both the scale and flexibility desired for all use cases.

Just as centralized compute never fully disappeared -- it took the form of client-server computing and the cloud, there's no question that centralized inference will continue to exist.

But it wouldn't surprise me if, ten years from now, 90% of global inference happens at the client.

While simultaneously, the 10% of inference that takes place in the datacenter is wildly lucrative and orders of magnitude greater than today.

Put simply: the inference pie may become so large that both client and datacenter win.

If you buy your own hardware, then after those four years you still have that hardware, and it likely still holds significant value. Plus you know the local model you're running today won't change or suddenly degrade in four months, let alone four years. Whereas after four years of paying subscriptions and API costs, you have literally nothing.