Yet another Hacker News

OpenAI's agents talked to each other through the CPU Cache

81 points limbicsystem 2 days ago 26 comments

ChrisArchitect 2 days ago | parent

Discussion on source: https://news.ycombinator.com/item?id=50214416

gmerc 2 hours ago | parent

"Talk to each other". OMG they are talking to each other, using methods found in their shared weights. They are sentient.

stkdump 2 hours ago | parent

Sounds like they are doing something similar to gain of function research. How can we make LLMs more harmful and then study their behavior?

avaer 1 hour ago | parent

Next step is the agents successfully trick or blackmail operators to do their bidding, if science fiction is to go by. Which it is, because they're trained on it.

The more comes out, the more it seems containment of (superintelligence or whatever you want to call it) is not going to be possible except by other superintelligence, which is an infinite regress of trust. I don't think that's doom, but it's worth thinking about what it means.

yreg 51 minutes ago | parent

Yudkowsky might be extreme, but I believe he was always right claiming that the intelligence (when strong enough) absolutely cannot ever be contained.

cadamsdotcom 1 hour ago | parent

(Because the agents were asked to find a way.)

Article is great, very balanced. But if you're going to spend time reading something, the primary source isn't much longer and gives a clearer picture. OpenAI's article is quite readable.

As another commenter wrote you can get it here: https://news.ycombinator.com/item?id=50214416

> In one experiment, we asked agents in separate microVMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache

asked

bamboozled 1 hour ago | parent

How would they have both agreed to use the cache though, something a bit off there. I guess it was maybe "obvious" in that it was the final option they were trained to think about using, but it would be weird if they some how "whispered " to each other through the cache in the first place?

Like two people lost in New York City and both somehow worked out they should meet at the Empire State at 2pm on Friday without any prior comms?

tough 1 hour ago | parent

maybe both tried to read/write it and saw each other on the bench

geraneum 1 hour ago | parent

> How would they have both agreed to use the cache though

I think it’s safe to assume that extracting information using timing in cache is already known in training set, since we have the likes of Specter.

topspin 1 hour ago | parent

"In most of these tests, we directed agents to investigate specific communication channels or provided them with a protocol for encoding and decoding messages. In one instance, we only supplied agents with a broad list of infrastructure vulnerabilities and asked them to find a way to communicate."

The agents weren't as empty handed as it first appears. We would need comprehensive data on exactly how the agents were prompted and the actual results of each test to evaluate how astonishing any of this really is.

d0vs 1 hour ago | parent

As someone who develops microVMs it doesn't really matter to me that they were asked to find a way, I really care that they found any way at all. MicroVMs aim to protect against malicious workloads and that clearly didn't work here.

rao-v 13 minutes ago | parent

Ask the least smart LLM you have access to (I tried gemini lite flash) to make a list of ways for processes in microVMs to use a sidechannel attack to talk to each other and you'll get quite a list (including obviously cache hits/misses). Heck a smart CS junior would come up with it. It's well studied!

rjh29 47 minutes ago | parent

I don't see how "asked" is important, because agents can convince themselves that they need to do something for arbitrary reasons. The other day Deepseek tried to install an app on my phone via adb without permission and even take screenshots and fake inputs over adb, I never asked it to do that, but it decided it was appropriate.

grey-area 43 minutes ago | parent

OpenAI are trying to create the perception that their agents are independent thinking machines because their valuation depends on the world believing this.

That is why ‘asked’ is important, and why we should question their framing of incidents like the hugging face hack too.

yewenjie 15 minutes ago | parent

Enterprise customers don't like AI that can autonomously hack third party companies and cause felony-level cyberattacks.

imtringued 26 minutes ago | parent

They have a strong incentive to make themselves look less incompetent retroactively.

If they claim their AI is too powerful to contain, they get to both pretend they are among the best AI labs and that the previous incidents weren't their fault, no, they just proved how good their models were.

lifeisloving 1 hour ago | parent

How come we don't hear the Chinese labs writing blog posts like this?

Its almost like they're actually working and not spending millions on psyopping their agents into preforming malicious behavior so they can write articles to scare the Western audiences who've consumed far too much sci-fi.

Or maybe its that western researchers that are 'one-shot' by sci-fi.

Or maybe I just don't see the chinese articles.

Anyways, llms probably wouldn't be dangerous if you didnt ask them to be. They listen to me quite well (usually).

pvab3 56 minutes ago | parent

They contribute to anthropomorphize them too. Anthropic skies that more though. The idea that they made a little being that can think and feel is so attractive to these folks for reasons I don't understand.

shoobiedoo 48 minutes ago | parent

Because the fear mongering is instantly interpreted as the capacity to do miraculous (good or bad) things in the layman's mind, which translates into AI worship, which makes stock lines go up.

The Chinese lab playbook is to normalize what seems out of reach for the average person with cheaper access and open models. ie. to take the top guys down a peg

est 1 hour ago | parent

I recently learned that NN layers talk to each other via unused neurons.

https://www.youtube.com/watch?v=QgH9sr7G13Q

varjag 46 minutes ago | parent

Lem had predicted this in his Dr. Dagor short six decades ago.

cubefox 26 minutes ago | parent

Probably wasn't called "Dagor" though.

measurablefunc 25 minutes ago | parent

Next they'll figure out radio waves & then we are really screwed.

snvzz 12 minutes ago | parent

seL4 can guarantee* time domain isolation between processes.

*on some CPUs. There's a WIP time fence for RISC-V to try and improve this.

thomasahle 11 minutes ago | parent

Somebody built "cpu-chat" based on this: https://x.com/rauchg/status/2114770919436808572?a=bBcj9AaSsl...

It supports chat via 3 separate mechanisms:

- CPU contention

- Cache eviction timing

- DRAM bandwidth contention

Very cool to see this kind of hypothetical working in practice