Yet another Hacker News

Kimi K3.1 scores 55 on Artificial Analysis, placed on the Pareto line

113 points theanonymousone 2 hours ago 32 comments

nektro 1 hour ago | parent

discussion being had on https://news.ycombinator.com/item?id=50212065

jawns 1 hour ago | parent

I know the log scale of that chart is valuable when surveying the full breadth of possibilities, but I would encourage people to make decisions between nearby models based on linear scaling. Linear cost is often more useful for making the actual decision.

ghasemi 4 minutes ago | parent

You may like this: https://aa.xlit.app/

dgellow 1 hour ago | parent

Looking at those charts makes me appreciate DeepSeek 4.1 flash even more. It’s just so cheap and fast while being a really decent model

probst 1 hour ago | parent

Would you please elaborate on "really decent"? Really decent for what use exactly? I want to love it (and use it as a cheaper research model in my harness), but I really cannot get any real development done with it? It stumbles and fumbles and over-complicates, where astra just plows ahead (often in less overall time) with excellent solutions.

I really want to love this model, particularly since it's good to have an open weight model on my team so to speak, but I think I might be holding it wrong.

What are you using it for?

esafak 1 hour ago | parent

Coding. What were you using last year? It's better than that.

andriy_koval 58 minutes ago | parent

Opus 4.6 was descent last year already

0xc133 45 minutes ago | parent

Anthropic released Claude Opus 4.6 on February 5, 2026. What year are you in, man?

andriy_koval 43 minutes ago | parent

thank you for nitpicking. Whatever was in Dec 2025 was very good already.

stymaar 29 minutes ago | parent

For agentic coding, Qwen 3.8-27B is not far from that, and Qwen 3.8 Next is there. Never used DeepSeek 4.1 flash but I'd be surprised if it wasn't better than this.

Onavo 52 minutes ago | parent

If you have well-defined, long running tasks that are relatively disposable but requires constant ML monitoring and overwatch e.g. web crawlers, monitoring prices, logs etc. DS4 Flash is perfect for those.

For tasks like crawling and real time log analysis etc., the costs really add up at scale.

samtp 47 minutes ago | parent

I can have 2-3 sessions running at the same time for pretty much the entire working day with DSF4.1 where the responses are fast, cost is pretty negligible, and the results are good enough 95% of the time.

On an overall speed/cost/quality comparison, it seems unmatched in my experience these days.

amelius 10 minutes ago | parent

Yes, people should say if they are using it for HTML/CSS/Javascript or Rust/C++; it makes a huge difference.

lnenad 1 hour ago | parent

Every single model I've tested that is supposed to be anywhere near Opus, wasn't there. There is some magic sauce in Anthropic models that these benchmarks aren't demonstrating.

mwigdahl 58 minutes ago | parent

Opus 5.5 is flat out amazing, particularly how cheap it is in practice (I'm on a subscription and use Claude Code and I almost never even approach the cap).

I would love a benchmark suite that better reflects real-world coding performance, as having Muse Wonder in the top three makes no sense to me.

weego 51 minutes ago | parent

I spent the last 4 months on a mix of chatgpt and glm building some prototypes of product ideas I've had for years because claude was in the phase of obnoxious conversation and verbosity.

I took the output and original specs and handed them to Claude today and the level of insight and concise detail in it's criticism and problem solving was fantastic. Dropped the others immediately

lnenad 49 minutes ago | parent

Same. Every time I prompt some other new/best/benchmaxxed model, I think to myself: wow, finally, this is amazing. And then I give the same thing to Opus and the bastard is always better. It's annoying to a point.

lnenad 49 minutes ago | parent

It's not just 5.5, every iteration of the frontier moving - Opus has had some edge. I'm sure it's not 100% in every scenario, I love interacting with Codex models more, but the quality of the output, in most scenarios that I've been exposed to, has been the best.

CuriouslyC 12 minutes ago | parent

Anthropic cooked with Opus 4.5 and 5.5. Their other iterations have usually been nicer to interact with than GPT and more "autonomous", while being dumber.

ozgung 49 minutes ago | parent

I think the secret sauce is about communication. It understands you and you understand it. Just a better coworker to work with. Benchmarks don’t measure that.

lnenad 26 minutes ago | parent

Yeah, but if you need to provide less details to Opus to get the same output it means the model is better at deducing your needs, which again implies it's a higher quality model.

mppm 44 minutes ago | parent

It's been suggested elsewhere that the AA-Omniscience benchmark [1] is more representative than the rest. It puts K3.1 about 6 months behind the frontier and the new version of DS 4.1-flash about 12 months. That roughly lines up with my gut feeling as well, at least when it comes to the trickier problems. Still rather impressive, but not quite frontier.

1. https://artificialanalysis.ai/evaluations/omniscience

jascha_eng 17 minutes ago | parent

+1 for the omniscience benchmark!

Although even this one needs to be taken with a grain of salt. Fable/Opus still act very differently to Astra and Argon imo. But maybe it's just because I am used to these. I wouldn't clearly put them above each other though, although I still prefer anthropics models, but that could be simply out of comfort.

logicprog 15 minutes ago | parent

Huh, this is an interesting hypothesis! It would make sense, to a degree. Part of "big model smell" to me is the model being able to pull a huge range of interesting algorithms, architectures, mathematics, and everything else that would come with greater world knowledge and having a much more sort of innate, like, full, well-rounded understanding of it instead of surface-level ICL pattern matching on web results.

lnenad 12 minutes ago | parent

Yeah that tracks completely with my experience as well.

dom96 42 minutes ago | parent

A lot of these models are likely benchmaxxing the existing benchmarks that are out there. Some perhaps even by accident.

I think looking at some more niche benchmarks helps, I created one and it does show Kimi K3.1 Pro as the most intelligent of all open-weight models, though it is still significantly below Opus 5/5.5[1]

1 - https://bench.killswitch-lang.org/

stymaar 33 minutes ago | parent

It really depends on the type of task you're doing and how you're using it, because for the past two month the only reason I've had to use my company's paid Opus plan instead of my local Qwen-3.8-Next was speed. I use Qwen as a back-up plan when my token limit is reached and in my day to day work they have been indiscernible in terms of output.

lnenad 28 minutes ago | parent

I agree, but my point wasn't about being able to use a weaker model, it's about benchmarks not being in any way representative of the actual quality/level of the model. Of course for a lot of scenarios Opus is overkill and Qwen/DS are more than enough.

CharlieDigital 8 minutes ago | parent

I think also heavily depends on what you're doing.

If you're in a stable codebase with lots of best practices, patterns, controls (linters, analyzers, etc.), examples already established, then any model will do ~the same plus/minus since there's not much novelty.

If you're building ground up, entirely novel, no reference, I think results will vary with lower capability models.

For me, I do a mix, leaving implementation to Haiku/Luna/DeepSeek and research/feedback/critique to Opus/Sol/Astra/Fable

rubinstein99 27 minutes ago | parent

Can confirm. Labs doing benchmaxxxxing and struggle to stay relevant

mijoharas 13 minutes ago | parent

Maybe I need to try it again, but I have never liked the opus models.

I keep having problems with them not doing the thing I ask, (and also failing to call tools correctly in pi. I think they've very much trained to the harness, which is frustrating.).

tonfreed 10 minutes ago | parent

I always found them too chatty and verbose. I'd get a wall of text and miss things because it was lost in garbage about load bearing and blast radius