70 points Retell15 1 hour ago 16 comments

bashhq_r 35 minutes ago | parent

how much vram do you need before qwen stops feeling sluggish on a normal laptop

Retell15 32 minutes ago | parent

You need more processor power.

steeleduncan 32 minutes ago | parent

which qwen model?

yread 5 minutes ago | parent

I run Basalt with Qwen 3.8 flash next (default Basalt NVFP4 quant) on 5070ti (16GB VRAM) with 96GB RAM. Prefill ~2000tok/s, generation ~100 tok/s. Definitely doesn't feel sluggish but it's not a laptop

IsTom 33 minutes ago | parent

Saying that 32GB+ VRAM is needed for Qwen3.6-35B@q4 is certainly an interesting statement.

steeleduncan 30 minutes ago | parent

That leaves 10GB(+) for context and KV cache, which seems about right

IsTom 29 minutes ago | parent

It's MoE and it runs on 3060 just fine.

baoooooooooooo 18 minutes ago | parent

Yeah, was this written before strata?

poulpy123 22 minutes ago | parent

Easy, you just need a 10 000€ graphic card !

angry_octet 18 minutes ago | parent

So pretty soon 10,000+ people could be running decent models on their home PCs with unsecured LAN access, connected to their residential modem? We won't just have residential proxy networks, we'll have residential ARTEX swarm clouds.

geraneum 10 minutes ago | parent

What do exactly expect to happen and how is it worse running other people’s agents on our machines?

amelius 15 minutes ago | parent

What do the local-llmers do to get an experience closer to Claude?

Do you reuse Claude's plugins/skill files?

Do you use a special harness or system prompts?

comprev 12 minutes ago | parent

OpenWebUI, OpenTerminal and OpenClaw are useful tools for using local models. There are dozens more tools available depending on your needs.

segmondy 11 minutes ago | parent

You can't get experience like clause but you can get result like Claude for 90% of tasks with more effort.

junga 9 minutes ago | parent

Set this up this weekend after AMD NPU support was announced for Unsloth. My ThinkPad x13 with the Ryzen AI 350 and 32 GB RAM is now serving my older GhostBSD ThinkPad running crush. Using Qwen3.5-9B (npu2) the quality of course is not comparable to the latest frontier models. Basically no result can be trusted. And 12 tokens per second are quite relaxing but it somehow feels 'right'.

snappr021 4 minutes ago | parent

Doesn’t sound to far off the simplexity chatted to ChatGPT and Claude yesterday.

When entire operating systems and software stacks to run end to end business processes, including engineering is a reality, Bill’s word “optimise” is close to the ultimate fulfillment.

There is too much compute in data centres, when multiple machines can be replaced with a functionally intelligent pi.