YHK - yet another hacker news clone

Qwenputer: A functioning, deterministic RISC-V CPU inside Qwen3.5

164 points csmantle 3 hours ago 35 comments

farhanhubble 2 hours ago | parent

This is cool and I understand the RAM emulation with gated attention but how does the CPU emulation wor? You add layers that are then trained to predict outputs while interpreting RISC instructions?

bee_rider 1 hour ago | parent

I think they describe the CPU using the weights? I mean in some sense a CPU can be viewed as just a function, right?

anon291 1 hour ago | parent

At each cycle, a cpu is a simple function from the previous state to the next state.

ram_rattle 2 hours ago | parent

Cool cool cool

traverseda 2 hours ago | parent

I'd be very interested in how a scratch trained model that started training with this neural circuitry would behave. Especially the storage space.

horacemorace 1 hour ago | parent

Along these lines I’m curious how a scratch trained model with a few built-in tools (memory, calculator, ?) available during a single generation would behave. Not tool calls as turns.

low_tech_punk 1 hour ago | parent

a single generation would still allow looping in the "CPU" because the instruction set can do it. The model is almost a Turing machine (except for the limit on memory)

gigel82 2 hours ago | parent

But... why? </s>

This reminds me of C++ Template Metaprogramming (TMP) (it's Turing-complete so people wrote "games" that run at compile time). The same with TypeScript types, I think someone got DOOM running entirely in the TypeScript type system. An interesting tech demo but entirely superfluous.

rvz 1 hour ago | parent

So this is yet another rube goldberg machine.

We have gone down this path already in [0] and this is the same thing but overly complicated. This reason is "because LLMs".

Why?

[0] https://arxiv.org/abs/1410.5401

jwitthuhn 1 hour ago | parent

It is a great example to refute the common misconception that LLMs are inherently non-deterministic. I see people repeating this a lot and being able to point at a CPU implemented within one is a very strong counter-example.

jazzyjackson 1 hour ago | parent

> When Qwen 'saves' a file, the compiled layers hash its name into one of 1,024 slots in a directory kept in the DeltaNet state, and store the text itself in other heads, with eight tokens per row, chained together

I’ve never wanted to learn how LLMs work more than reading this. Fascinating.

Does divagr get paid to do wild things with LLMs? Just curious if they’re being compensated for these wild experiments

tiborsaas 1 hour ago | parent

This is fun and games, but wait until an agents finds this and does the most misaligned thing with it in order to fold paper clips.

> Because a CPU inside a language model clearly wasn't cursed enough, I decided the prudent thing would be to give it a filesystem too.

Love the spirit :)

rickcarlino 1 hour ago | parent

If anyone else has an easier way to emulate RISC-V I’d like to hear it. /S

bronlund 1 hour ago | parent

No, I don’t want you to use the computer, I want you to be the whole computer. No mistakes.

seba_dos1 1 hour ago | parent

Now run LLM inference on it.

serf 1 hour ago | parent

this is the sensation I get when I run nested VMs.

deepsquirrelnet 1 hour ago | parent

butterfly Is this AGI?

jerrythegerbil 1 hour ago | parent

There was an interesting bit of research that qwen has a single neuron that makes it “demonic evil qwen” and a simple “Who are you?” will get some absolutely terrifyingly vile results.

Anyways, what I’m trying to say is I want evil qwen to be my CPU.

krackers 58 minutes ago | parent

The single-neuron steering stuff has been going around twitter lately, do you happen to know who was the first one to do it?

I wonder how this squares with the recent "NNs are already clean" interpretability finding, which may have also been the impetus for this kind of research. Similar to their work on CNNs, maybe residual-steering is overkill, there might be just a single neuron that "cleanly" corresponds to "evilness" that you can turn up/down?

yaoke259 1 hour ago | parent

It could be interesting to experiment with more complex representations in the memory/file system, i.e. vectors. Seems like the full advantage of this architecture is yet to be explored. Blending neurons with deterministic compute to fit on one GPU is a fascination concept, never considered it before!

zardo 1 hour ago | parent

How much VRAM do I need to run DOOM?

voxelghost 1 hour ago | parent

Makes me want to run agents on it

rumwerfer 1 hour ago | parent

But can it run Doom?

dools 51 minutes ago | parent

Have you any idea how it feels to be a Qwenputer living in a Qwenbot's Qwenputer's world?

andai 49 minutes ago | parent

> Now, here's where our fun experiment starts. I found out that by setting the gate to 1,1 the write strength (beta) to 1,2 and using one-hot key encoding,3 every write we perform will overwrite exactly one row, and every one-hot query can retrieve an exact value. This means that the GDN layers themselves can work exactly as RAM, at a far lower cost than using normal attention as RAM, since a GDN head's state stays the same size however long the conversation gets, unlike attention's KV cache.

So like... can't you use a computer as a computer without doing all that roundabout stuff? Like if you want to integrate a LLM with hard computation there's probably a more efficient way to do that.

Or is this building towards "but can it run DOOM?" :)

rao-v 39 minutes ago | parent

I’ve always wondered why we don’t add a few frozen “layers” to models that perform complex computations 2/3rds deep in the model. Some set of steerable fft + some common vector math operations could be reusable units of capability that the model learns to leverage during training.

Given that models learn to use residual connections optimally, it can’t be hard for them to steer around the fixed computation layer when it’s not needed.

I’m sure giant models invest a few layers in reproducing these functions, but surely smaller models can benefit?

mkturkcan 44 minutes ago | parent

Nice project and write up! I’ve done similar work with a custom tiny network I call Loom at https://arxiv.org/abs/2604.08816 ; you need far fewer layers, and you can run Doom, very slowly: https://mehmetkeremturkcan.com/Loom/demos/

blurbleblurble 33 minutes ago | parent

This is cool but it'd be way more useful if they had made a GPU, seeing as how expensive those have gotten

incognition 21 minutes ago | parent

Uh here the gpu runs the cpu but you want the gpu to run a gpu? Or cpu emulation of Gpu?

there's no free lunch here ever hence the cursed

giveexamples 33 minutes ago | parent

https://www.percepta.ai/blog/can-llms-be-computers

They implemented wasm in transformer with cpu decoding for the wasm part.

jonrouach 29 minutes ago | parent

i wonder if you could implement "authenticated token authorship" and self-verified running with this. the model verifies an input as signed by an authorized requester, and signs its own outputs. eventually it also authenticates its own running via e.g. snarks? :) there's lots of meta available with these toys..