78 points Philpax 1 hour ago 30 comments

wmf 1 hour ago | parent

Congratulations I guess but it's a pretty bad look to not even mention those other open weight models.

flatline 58 minutes ago | parent

Right? These are not likely to be meaningful points of comparison to anyone but the authors.

wmf 51 minutes ago | parent

Maybe Palantir will use it or something. There probably is a market for US models but I don't know how large it will be.

Onawa 37 minutes ago | parent

US Gov won't let Chinese models be hosted on any internal infra, not even supposed to use them for local AI. So there is absolutely a market for US-made or non-Chinese models there.

tedivm 58 minutes ago | parent

We've always been at war with eurasia.

Seriously though, it's just easier to pretend that China doesn't exist when they're stomping us this badly.

karmakaze 44 minutes ago | parent

What's funnier is that after cherry picking models for comparison, they further cherry picked Inking for size comparison instead of better and smaller Beam 1.0

The bar chart also shows Nemotron 3 Ultra instead of smaller and better Nemotron 3.5 Super.

petcat 4 minutes ago | parent

Do those other models usually mention all the competition? I haven't noticed

RGS1811 1 hour ago | parent

The comparison chart is befuddling.

dofm 1 hour ago | parent

So we have a Recursive and a Ricursive, both making LLMs?

xyzrt 55 minutes ago | parent

One making self-improving AI via AI-chip co-design, the other relying only on software for self-improving AI.

dofm 54 minutes ago | parent

I hope they never have to call each other about anything. It could get confusing if neither knows who is the base case.

giwook 50 minutes ago | parent

There seems to be some recursion going on here.

wmf 49 minutes ago | parent

Time to start Rickursive, the lab that will never give you up and never let you down.

throwaw12 32 minutes ago | parent

we need RichUrsive

6thbit 23 minutes ago | parent

And it should acquire the existing two

neutronicus 8 minutes ago | parent

Roll them up, you might say.

syntaxing 59 minutes ago | parent

When a post doesn’t mention Qwen for comparison, you know the performance isn’t going to be great.

ethan-boneh 54 minutes ago | parent

I mean Qwen 3.8 27B hits 73% on TB2.1 and 42.2% on DeepSWE....

2001zhaozhao 53 minutes ago | parent

This model scores 80.5 on Terminal Bench 2.1

For reference Qwen 3.8 Flash Next (which is bigger than this model but probably cheaper to run due to MoE) scores 84.3, and Qwen 3.8-27B scores 73.0.

I think this model could be competitive, depending on how much it was benchmaxxed compared to Qwen?

advael 52 minutes ago | parent

Category narrowing to this degree really has diluted the term "SOTA" quite a lot

You mean it's state of the art for this subset of benchmarks you're targeting? No? Well surely it's state of the art for models of a certain size category? Huh, what about models that are available for local inference? O, state of the art for open models of a certain size made in the US specifically. Uh. Neat!

visarga 36 minutes ago | parent

No, the term is well defined. It means being on the Pareto curve of performance vs size, such as: https://tinyurl.com/26sw35fw

O_H_E 20 minutes ago | parent

well yes but the y axis heavily depends on the benchmark you choose

advael 1 minute ago | parent

Irrelevant. A term can be well-defined and used correctly while also being diluted. In this case, I am pointing out that it's diluted by narrowing the category in which the claim is being made. I have no doubt that in this narrowed category, the term is being used correctly. This has no bearing on the fact that by narrowing the category in that way, this claim is made remarkably less impressive or interesting to many people hearing it, including myself

jszymborski 50 minutes ago | parent

Would love a 27B model. Wonder if we'll get a distillation.

6thbit 27 minutes ago | parent

I'm sorry but what is "U.S." implying here?

Is this sponsored by the us gov? did it only train on US data? does it only work on the US?

conception 26 minutes ago | parent

It's a US group that made it, in the same vein that OpenAI and Anthropic have US models.

petcat 24 minutes ago | parent

It means that it's an American AI lab behind it.

soundworlds 19 minutes ago | parent

Not to be confused with https://recursive.com/

johnnyApplePRNG 11 minutes ago | parent

Only benchmarking against other garbage american models?

LMFAO get out of my feed

stewartpark 8 minutes ago | parent

It's great to see another competitive open weight model. Congratulations!

But as is, it's probably hard to become popular amongst hobbyists and local model enthusiasts like Qwen's 27B class models or MOE models because it's a 65B dense model and inference will be prohibitively slow on consumer grade hardware, and seems like the goal/mission of the company behind this model is to automate chip design.

So if I have to assume and connect the dots, this is for the companies who want to use AI in air-gapped environments for chip design so that they protect their IPs (not leaking to third party AI labs/hostile parties) and can afford lots of server grade inference hardware for a quick unlock with the big dense model.

Neat!