YHK - yet another hacker news clone

A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices

247 points acossta 4 days ago 100 comments

Bric3d 4 days ago | parent

They'll throw this together but still not sell you something you can stick in a rack ?

Telemakhos 4 days ago | parent

That seems to be the point: you leverage existing workstations on your corporate network and run compute batches at night while they're idle. You don't have to buy any extra servers for your low-priority AI workloads.

jprd 4 days ago | parent

We were doing this decades ago with engineer's HP C3600s under their desks. It wasn't AI, just low-priority HPC / fluid stuff batched out to hundreds of workstations.

macintux 4 days ago | parent

That was the original pitch for Ian Murdock's Progeny startup in the late 90s: a Linux-based distributed filesystem for corporate desktops to boot into overnight for distributed computing.

mgerdts 3 days ago | parent

And on IRIX, SunOS, and DG-UX boxes too. Once Linux existed for a bit, they were added to the fold. The software to do this - Condor - is still maintained.

https://pages.cs.wisc.edu/~danb/ref-v7.7.5/ref.html

simbas 4 days ago | parent

That's why the button is on the underside.

sdoering 4 days ago | parent

Pardon me not understanding what this is about?

Basically SETI for AI workloads? At least, this is the impression the landingpage left me with.

I am a bit lost, tbh.

railka 4 days ago | parent

Minions is an internal enterprise system already in use at Apple. It pools idle Mac GPUs into a shared compute cluster for ML inference.

There is no distributed execution of a single LLM across multiple GPUs using tensor parallelism here. Instead, it relies on task-level parallelism-breaking down a large workload into independent subtasks that run across separate machines. For example, generating embeddings for a million documents.

alt227 4 days ago | parent

So basically works the same way as Folding@Home?

rramon 4 days ago | parent

Only on the same local network I guess.

rafaelmn 4 days ago | parent

Embedding@Work

cr125rider 4 days ago | parent

HotBackpack@Commute

1e1a 4 days ago | parent

DeadBattery@Work

coob 4 days ago | parent

or distributed.net

greggsy 4 days ago | parent

My university ran its render farm across idle workstations using the screensaver back in early 2000. Same concept.

pjmlp 4 days ago | parent

Psst, it is "innovative". /s

gizajob 4 days ago | parent

It is if it’s GPU compute.

pjmlp 4 days ago | parent

Distributed GPU compute has existed for quite some time,

https://developer.nvidia.com/gpudirect

And this is because I can't be bothered to dive into older papers, so recent NVIDIA stuff will do.

spacedcowboy 3 days ago | parent

Not saying you’re wrong, I agree that distributed GPU is available* but that link you provide seems to be for GPU<->Storage, not computer.

*: hell my own compiler is moving to multi-GPU on a single computer and networked devices are the obvious next step.

juliendorra 4 days ago | parent

And Apple has had simple distributed video encoding since 2005, so yes it is not new in computing, but it is still useful for the gap before a given task can fit comfortably on one machine

dekhn 4 days ago | parent

This pattern has been implemented across hundreds if not thousands of different organizations (and between organizations). I have a patent on data distribution strategies from my time at Google running a large (>1M cores 24/7 for 3 years) idle cycle harvesting project.

it's a good idea but you often end up with huge tail latencies, slight incompatibilities due to fleet heterogeneity, and a database/scheduler management problem.

veexx103 4 days ago | parent

This project sounds good.

What is it actually like?

tom1337 4 days ago | parent

I guess for companies this can be nice? Run some inference in the off-hours on the machines that you already have. Could be useful for analytical tasks that don't require to be instant. On the technical side this doesn't seem that groundbreaking for me though.

pm90 4 days ago | parent

My warm take is that Apple is leaving a lot of money on the table by not investing in creating an independent supply chain for their chips and building their own cloud. They arguably had the first mover advantage when it came to general purpose arm compute; all they had to do was flood the market and make it easy for anyone to rent it. Not only would they corner a part of the market but perhaps ease demand pressure on other chip manufacturers too.

sigmoid10 4 days ago | parent

Apart from straight up buying TSMC's or Samsung's fabs I don't see a way of building a vertically integrated high-end chip supply chain for Apple. At least not on timeframes measured below a decade. And on those time frames the current supply issues will probably be resolved by companies who put all their effort into it instead of just tweaking their bottom line.

alt227 4 days ago | parent

TBH Im surprised they havent already bought one of these companies.

macintux 4 days ago | parent

They've had a great deal of success playing them off against each other.

r_lee 4 days ago | parent

because their entire thing is to focus on a few product lines... I'm not surprised at all

bel8 4 days ago | parent

I don't think they would be able to buy any of these. It would have been blocked by multiple regulators.

bensyverson 3 days ago | parent

I think they may eventually be able to buy the US TSMC facilities and assets.

ecshafer 4 days ago | parent

There is zero chance Taiwan or South Korea would allow this. It would be blocked on National Security grounds for each respective company/country.

jacquesm 4 days ago | parent

It might actually serve as an insurance policy of sorts.

lostmsu 4 days ago | parent

Does not seem to be helping AWS data centers in Middle East.

nxobject 3 days ago | parent

I actually see the logic there: Apple as a “harmless consumer device company” is probably one will remain very geographically diversified in the long run, between the China and America markets.

(Interestingly enough, I think PRC and ROC are combined in public financials.)

reachableceo 3 days ago | parent

Why would they want that on their balance sheet?

Also they have the ability to put in sufficiently large amount purchase orders to buy up whatever capacity they want.

Even at ludicrous markup prices from TSMC, ifs going to be orders of magnitude cheaper than in house a fab.

pjmlp 4 days ago | parent

They don't get enterprise, A/UX, mkLinux, OS X Server, Mac Pro tower, all gone now.

Swift's main reason to go open source and have good Linux support was naturally for app developers to be able to use it on server side, which naturally wasn't going to be macOS.

I macOS wasn't required for developing iDevices apps, they would surely sell even less desktops.

nxobject 3 days ago | parent

If anything, Apple has every incentive to hop on the Linux Foundation train like MS. Just for containers, of course ;)

yohannparis 4 days ago | parent

My guess is that it's already hard for them to get as many chips fabricated for their devices, which is still their main source of income so far.

Maybe their chip team is already on the work, or already made, a server chip for AI purposes used internally, and doesn't have to talk about it yet.

kolinko 4 days ago | parent

My guess is that they have challenges scaling the production anyway (6-16 weeks wait time for Studios), and also they would leave money in the table if they just released chips. I would assume that once they scale the production they will get back to the server game - MacOS server racks, or something more inventive like MacOS supercomputers running on a ton of prev gen chips ;)

curt15 4 days ago | parent

They'd be left maintaining their own macos server again while everyone else uses Linux for their server farms.

nxobject 3 days ago | parent

At this point, I’m sure bankrolls a significant amount of R&D in new MSes and build-out capital, in return for guaranteed capacity on the first production lines. I’m not sure the extra juice of outright ownership is worth the squeeze.

(As much as conglomerates are fun…)

gempir 4 days ago | parent

Interesting, so this is basically Apple's version of exo ? https://github.com/exo-explore/exo

skhameneh 4 days ago | parent

Reading the “paper”, this is just a job runner built on Ollama. Exo, and the teams at Apple that built the foundations Exo uses, actually use modern distributed compute.

jacquesm 4 days ago | parent

I don't get why they didn't use llama.cpp instead.

brandonpelfrey 4 days ago | parent

I don't get it. In the paper it sounds like it's "just" a CRUD DB, then you have some fleet of Macs, and they store inputs and outputs in S3. What's the innovation here? Not trying to poo on this, but it looked like the most basic and obvious approach.

pjmlp 4 days ago | parent

Apple rediscovering HPC clusters, but without OS X Server to serve alongside it.

skhameneh 4 days ago | parent

I’m a bit disappointed in the lack of depth (and use of Ollama): https://arxiv.org/html/2610.12970v1

Apple has released some small models previously that seemed genuinely great, along with some quality research/efforts. I’m just not seeing the same level of depth here, am I missing something?

serhack_ 4 days ago | parent

true, shame on apple for the paper quality

neuronexmachina 4 days ago | parent

I think it's basically just a write-up of something they're already using internally.

tw1984 4 days ago | parent

internal system, so the IT/HR department is the solution to leechers.

nothing new here.

djoldman 4 days ago | parent

Paper:

https://arxiv.org/pdf/2610.12970

Shortest promoted paper I've seen from a FAANG.

sehw 4 days ago | parent

>Apple steals your electicity while you take a dump

lynndotpy 4 days ago | parent

It would be so incredibly cool if we finally had an accessible consumer-accessible way to turn a network of computers on a network into a supercomputer. A household has a child who gets into 3D artwork and wants to render something? People can opt their tablets, phones, laptops, whatever to crunch out some frames overnight while not being used.

botanrice 4 days ago | parent

until you get a teen obsessed with 3d rendering then they're up all night finding a way to convert manga into 3d scenes and your old laptop is running 100% cpu trying to keep up. xDD

lynndotpy 4 days ago | parent

The idealized version of this in my mind has a consent-first interface which makes it easy to set boundaries and limits and wouldn't burn out the machine as a default

twosdai 4 days ago | parent

This is basically what seti and the computing @home projects did. Just doesn't get as much support outside of hobbiest and institutions.

lynndotpy 4 days ago | parent

Yes! I remember it being a bit difficult and confusing to set up when I was 10 or so. And of course, you can't use it on e.g. your gaming consoles or phones, and AFAIK you can't set up BOINC for personal use (e.g. rendering your own blender projects on all your machines). (That said, Blender itself does provide good tooling here for Blender specifically).

willquack 4 days ago | parent

There is a startup working on this called Distributive I used to work for - their compute nodes are based on js/wasm runtimes so you just need a web browser to turn a device into a compute node...

reachableceo 3 days ago | parent

Didn’t someone make a k8s worker in JavaScript a few months back?

chasil 4 days ago | parent

I'm not comfortable giving away time and power consumption on a machine that I own.

I would definitely opt out.

Is Apple going to compensate the owners?

bee_rider 3 days ago | parent

Maybe users could generate compute credits by taking tasks, and cash them in by submitting them.

elcritch 3 days ago | parent

> if we finally had an accessible consumer-accessible way to turn a network of computers on a network into a supercomputer

There's plenty of devices operating in giant supercomputer networks accessible for cheap to consumers! Unfortunately the consumers in question are spam farms, bot networks, ddos rings, and other mostly criminal types.

It would be cool if similar networks could be used for good instead. Earning credits providing compute that could redeem for spurts of fast parallel compute.

mindwork 4 days ago | parent

Is this opt-in or opt-out?

cousinbryce 4 days ago | parent

I’ll opt-in to working at Apple

eikenberry 4 days ago | parent

Sounds similar to Salad (salad.com), except for the differing hardware.

magicmicah85 4 days ago | parent

It's not a novel idea, but its provable that someone can take this same concept and do something truly groundbreaking. I wonder if there's value in the idea that you can join a supercomputer network of MAC/Nvidia GPUs as a way of getting off cloud and into "our cloud". As long as you offer compute, you have credit into the system to be able to support your own workloads that can't be done on just your hardware. Need to address concerns about privacy, abuse, etc, but Apple's idea here has some merit if applied right.

nico 4 days ago | parent

Would love to build something like this but for classification models that run on CPU. Like a “Classify @ Home” p2p compute network, to which people could contribute their machines and see what work is being done

Building the classification part now :) https://jeffyclassify.com/

bouk 4 days ago | parent

Why is this a paper? Should've just been a blogpost. Maybe I don't understand academia

elcritch 3 days ago | parent

Paper == citations

cmdrk 4 days ago | parent

Apple has reinvented cycle scavenging about 40 years later.

jacquesm 4 days ago | parent

There are a lot of parallels between the DGX (and clones) series devices and these Apple devices, maybe it would be possible to rope in the DGX boxes as well. There's lots of them out there and I'm pretty sure not all of them have high utilization.

MomsAVoxell 4 days ago | parent

The DGX are certainly making a case for a new workstation platform to compete with Apple. The user experience is on par, and the local model capabilities are very appealing in many use cases. I've been happy to have had a pair to play with anyway .. I sure hope the price comes down and Nvidia expand the concept ..

jacquesm 4 days ago | parent

For now the prices seem to be only going up. It is complete madness. At the same time: I'm kind of happy about that. It means that lots of people and small to mid sized organization have a very nice incentive to make low end hardware perform better. If you look at how much effort is being poured into model compression and clever VRAM management schemes on the low end it is quite impressive. And plenty of those are compounding.

I wished they had a bit more memory bandwidth though, this is quite annoying.

d--b 4 days ago | parent

If I can get paid for heating my home with LLM queries this winter, I'm all for it.

mcculley 4 days ago | parent

As with all similar SETI@home approaches, this paper does not address the energy cost.

Yes, we can distribute tasks to idle machines in homes and on office desktops, but it is often much more expensive than sending it to a data center where power is cheaper and the hardware is more efficient.

BoppreH 3 days ago | parent

Might have an edge in winter, when you're paying to warm up the room anyway.

elcritch 3 days ago | parent

Power usage used to be more negligible back in the @home projects heydays. They had some cool screensavers too.

testfrequency 4 days ago | parent

I built something very adjacent to this right before leaving Apple, same name, handed off to my team.

I’m very curious if that project evolved to this……..going to message a few friends now heh

buildbot 4 days ago | parent

We did a little demo/have a patent for a similar idea using idle GPUs on Mercedes vehicles once upon a time - https://patents.google.com/patent/US20230281046A1/en

willquack 4 days ago | parent

Imagine if Apple made a GENERIC distributed compute framework to leverage idle computing power from Macs for distributed Rendering/Compiling/etc use cases. I'm sure Apple could convince Adobe and others to support this framework for their compute-intensive products and create a really interesting ecosystem for it.

Then universities and business could reduce their need for cloud and have a reason to put that spending towards higher end Macs...

yardie 4 days ago | parent

Literally built into it since 10.6.

https://developer.apple.com/documentation/dispatch

wtallis 3 days ago | parent

There's nothing distributed about GCD; it only works within a single machine.

yardie 4 days ago | parent

If you've been around Macs long enough this is very much in Apple's wheelhouse. They already do something similar with Final Cut Pro rendering, Logic Pro, Grand Central Dispatch, and XCode distributed CI/CD. Apple HQ is probably sitting on 1000s of nodes, high powered MBPs, iMacs, that could do lots of inference work overnight.

[0] https://en.wikipedia.org/wiki/Grand_Central_Dispatch

[1] https://developer.apple.com/documentation/dispatch

nsagent 4 days ago | parent

Reminds me of Incredibuild.

pjmlp 4 days ago | parent

Available by default on Visual Studio, just as info.

javiramos 3 days ago | parent

and Folding@home

saagarjha 1 day ago | parent

I don't know why you are linking those, they have nothing to do with distributed computing

splittydev 4 days ago | parent

Darkbloom is already doing that today (darkbloom.ai)

janandonly 4 days ago | parent

I remember seeing this at a presentation two or three years ago. I’m surprised it wasn’t rolled out right after?

nsagent 4 days ago | parent

  Future work will extend Minions to support distributed model training in addition to inference
That could allow enthusiasts to pool compute resources on devices like Mac Studios, Mac Minis, etc to perform large scale federated training. While it'll be massively inefficient compared to running in a data center, it could democratize open weights models even more.

It would be especially useful for RL post training of existing models, which is where much of the agentic abilities arise from.

curt15 4 days ago | parent

Something like the old folding@home distributed molecular dynamics calculations?

ElijahLynn 3 days ago | parent

Millions of Mac users could make passive income renting out their plugged in GPU and compute...

1-6 3 days ago | parent

Sounds like Elon Musk's plan of Tesla owners selling idle compute on their cars.

bigyabai 3 days ago | parent

No, they really can't. There is no market for Apple Silicon GPU compute, largely because the fastest Mac GPUs you can buy are arguably a waste of TSMC's silicon.

Look at it like this: a maxed-out M5 Ultra Mac Studio starts at $6,799, and the iGPU benchmarks similar to an $800 RTX 4070 Ti SUPER. That's a 3nm SOC getting outperformed by a 5nm dGPU; Apple Silicon is years behind Nvidia's consumer offerings, and that's before we even compare with datacenter-grade DGX/Instinct GPUs.

sleepybrett 3 days ago | parent

sorry $1300 for that 4070 now

badc0ffee 3 days ago | parent

The combination of "maxed-out" and "starts" is kind of confusing, but I found the machine you're talking about - M5 Ultra with the 36-core CPU and 80-core GPU, 96 GB RAM, 1 TB SSD.

It has 6x the RAM, 2x the memory bandwidth and looks like it totally outclasses the RTX 4070 Ti SUPER. I did some searching and it looks like a close comparison would be the older M3 Ultra Mac Studio, although again, with a lot less RAM.

bigyabai 3 days ago | parent

OP's claim was that the GPU could be rented out. I'm calling it like it is, there is no market demand for that anywhere in the world.

Don't take my word for it; go look at the number of datacenters that offer MLX or MPS inference. It isn't a market, nobody wants these GPUs even during a RAM shortage.

luizfelberti 3 days ago | parent

This reminds me of this blog post[0] where they intercept calls to the CUDA API from one host to execute them in a separate host, essentially giving GPUs to VMs that have none, and allowing one GPU to serve multiple instances.

[0]: https://loopholelabs.io/blog/ollama-remote-gpu