YHK - yet another hacker news clone

Unsloth Qwen3.8-27B Quantizations – Dynamic v3.1 Update

44 points Retro_Dev 4 hours ago 5 comments

jan_Sate 2 hours ago | parent

Awesome! I wonder how it does at Q5+ compared with v3.0

ilaksh 1 hour ago | parent

How do the smallest quants of this compare to Bonsai 2?

DiabloD3 1 hour ago | parent

Bonsai 2 doesn't work at all, Unsloth's smallest... also doesn't work, but is still better.

Bonsai's technology would be better suited for 100s of B models.

nout 1 hour ago | parent

How does it compare to the 3.8-27B versions from TensorFold? I.e. does it make sense to run unsloth via TensorFold engine?

aliljet 18 minutes ago | parent

There are so many of us with weak GPUs and these updates are massive wins. Still waiting for the 2080 moment where this 27B param model and a reasonable kv cache can exist and function at slow, but overnight-burning speeds ..