YHK - yet another hacker news clone

MiMo-v2.6-Pro: Intelligence, Performance and Price Analysis

119 points theanonymousone 10 hours ago 46 comments

jampekka 8 hours ago | parent

"When evaluating the Intelligence Index, it generated 140M tokens, which is somewhat verbose in comparison to the median of 140M."

throwa356262 8 hours ago | parent

Nowadays these error can be a good thing :)

Human error means this wasn't just stopped together by some bot.

jampekka 7 hours ago | parent

My bet is that it's a bot error, but of a rule based one.

theanonymousone 7 hours ago | parent

Yes, it seems they have a template that they fill with numbers. Similar issues spotted on Grok's performance page: https://news.ycombinator.com/item?id=49789558

MichaelNolan 22 minutes ago | parent

I have seen them say "and reasonably priced when comparing to other models of similar price" on several models. I always get a kick out of it.

egeres 6 hours ago | parent

It feels suspicious that MiMo-V2.6 Pro gets 46 in de index while DeepSeek-V4.1 (https://artificialanalysis.ai/models/deepseek-v4-1-flash) gets 39. According to the appendix at the bottom of https://mimo.xiaomi.com/mimo-v2-6 the deepseek model sometimes surpasses mimo and it's not so far behind in capabilities. A week ago opus 5 appeared 1 points ahead of fable 5 despite fable being a much smarter model (this has been corrected already)

GodelNumbering 5 hours ago | parent

> It feels suspicious that MiMo-V2.6 Pro gets 46 in de index while DeepSeek-V4.1 gets 39.

Why?

big-chungus4 2 hours ago | parent

> W

W what?

SyneRyder 5 hours ago | parent

The main AA benchmark keeps changing, and had to be radically changed when Astra came out and showed zero improvement over GPT 5.6 Sol in their benchmark. Opus 5 is still 1 point ahead of Fable 5.0 on the index, if you manually add Fable 5.0 back into the list, so it hasn't actually been "corrected". It's only Fable 5.1 that is shown as ahead of Opus 5.

The AA benchmark is a weighted average of other benchmarks and some internal ones. I think the difficult part is finding benchmarks that reflect your own use of the models.

seahorseemoji 3 hours ago | parent

The way Artificial Analysis keeps changing their weights feels kind of like deciding who the winner should be and making the weights reflect that. They’ve been changing their weights to add more weight to improved long-running agentic capabilities, but doing so means they’re reducing the relative importance of world knowledge and of writing ability.

I’ll grant that maybe world knowledge isn’t that important for these models. But writing ability is important for human understanding, and I think the weird turns of phrase and word choices reflect the labs’ underweighting of the importance of human understanding.

sipjca 1 hour ago | parent

I mean artificial is in their name....

conception 1 hour ago | parent

DS 4.1 is good but it’s clearly not as “smart” as non-flash models- it just doesn’t have the training data. Without a solid plan, it goes off the rails pretty regularly.

dom96 6 hours ago | parent

It is an impressive model. Agreed on most that is written on this page, with the exception of it being fast. I ran it on my own LLM benchmark suite[1] and it is faster than DeepSeek but still much slower than leading models. But it's pricing is where it really shines.

KillSwitch-Bench 1.0

  Claude Opus 5           66.9
  GPT-6 Astra             57.9
  Claude Fable 5.1        46.7
  MiMo-V2.6-Pro           38.8
  Muse Spark 1.3          36.5
1 - https://bench.killswitch-lang.org/

ricardobeat 3 hours ago | parent

Speed seems to vary a lot with demand. Last night it was reaching 80+ tok/s

drittich 1 hour ago | parent

Interesting - have you done MiMo-v2.6-flash?

conception 1 hour ago | parent

That looks like you aren’t using the ultra speed endpoint.

gandreani 35 minutes ago | parent

Why does Luna have a score of 0? I will say that in my limited experience, I use Luna and DSv4 Flash (haven't tried 4.1 yet) and Luna is wayyyy faster. They both output at the same speed but DS has an endless thought process

kosolam 6 hours ago | parent

Why sol is not in the comparison?

guelo 5 hours ago | parent

the graph has a dropdown for selecting models

kosolam 3 hours ago | parent

Not on mobile unfortunately

guelo 3 hours ago | parent

Not the graph at the top, the one further down.

Gareth321 6 hours ago | parent

OpenAI usage limits have been severely cut, and intelligence appears to be markedly declining, so I'm going to start trying these Chinese models seriously now. I don't mind if it takes longer. I just need the intelligence to predictably work the same way from day to day.

unsupp0rted 5 hours ago | parent

Same- I pay $200/mo for Codex but whereas I used to get a week's work out of a weekly limit, now I get roughly 1~2 days.

I've stopped using Astra entirely and remain on Sol orchestrating Luna Xhigh, but it's still not nearly a week's usage for a week's allotment.

And even then, whenever a new model is about to come out, it feels like the model I'm using is being dumbed down substantially.

I have no evidence for this and can have no evidence for this, but I can vote with my wallet regardless.

Gareth321 5 hours ago | parent

I strongly agree. Check out the Codex subreddit. Many empirical examples of Astra silently downgrading the models. One found Astra was silently using Luna Max (but still billing for Astra).

Even when I try to stick with Sol X/High, my limits are at best half of what they were before Astra launched, and the intelligence has declined markedly.

I cancelled my $100 plan. This is absolutely absurd and frankly unusable now.

Muromec 4 hours ago | parent

It feels bizarre reading about the amounts spent on it here and paying like 10 eurobucks a week for DS

Gareth321 4 hours ago | parent

These tools were pretty great if you could afford them, but now they are expensive and shit, and that combination doesn't work.

f6v 3 hours ago | parent

For me, DS Pro is still behind Sol. But I do think many people hitting the Codex limits in 1 or 2 days are doing something wrong.

sjbzbeiks 1 hour ago | parent

Entirely depends on the work asked and the repo involved.

When I’m doing work on a repo where I’m implementing a standard and the agents have to read the standard to keep from hallucinating my usage skyrockets.

Hell this changes depending on which language I’m working with.

christophilus 2 hours ago | parent

My spending with Deepseek was more than a Codex subscription, though. How are you keeping it to $10/month? Super light usage?

dangoodmanUT 1 hour ago | parent

see that's interesting, because I'm prompting all day and I usually end up with 15-25% by the end of the week. I'm using Astra xhigh exclusively

sauwan 58 minutes ago | parent

Weird. That's definitely not my experience. On the $100 plan and I can burn my entire week's budget in a few hours easily with Astra. It's borderline unusable.

gandreani 39 minutes ago | parent

Are you using codex or another harness? The problem could be the harness or using plan mode.

I wonder what dangoodmanUT is using! This is the time to compare!

phoghed 3 hours ago | parent

> and intelligence appears to be markedly declining

Serious question: does anyone have evidence of this?

It’s something that’s constantly asserted, and has been since 2023. Every time someone posts a site that tries to track this though, I look at it and it’s just a flat line.

conception 1 hour ago | parent

https://marginlab.ai/trackers/codex/

By and large they don’t. I have seen this drop a few times, eg before fable came out opus dropped a lot probably due to less compute available.

My guess is it’s a combination of getting used to the new cliff models fall off on and forgetting that model performance drops significantly when context fills up.

So new model comes out, people try it and it’s amazing on a task or two. Then they start using it, context window fills up and it gets a lot worse.

loloisi 3 hours ago | parent

Sol 5.6 xhigh had been a very reliable workhorse for coding for me via the 200 bucks sub.

But this week they seem to have tweaked the system to a point at which all models (Astra, Sol, Luna) hit rate limits all_the_time without me being anywhere close to the weekly limit.

Early results with MiMo 2.6pro are quite encouraging for anything that's non-UI work so likely switching spend for the time being

ignoramous 5 hours ago | parent

Per Xiaomi, MiMo v2.6 training run cost $3.47m. A far cry from the estimated costs ($100m+) for the Big 5 (MSL, xAI, GDM, OAI, Ant). I wouldn't be surprised if salaries and R&D costs have similar drastic disparities.

For a model that matches Muse Spark 1.3 in benchmarks, MiMo v2.6 Pro is incredibly cheap, given its cache rates will remain $0.0036 per million.

NortySpock 5 hours ago | parent

I sorta got the impression that the $3.47 million only covered post-training , given that few of the graphs start at zero. Is a barely-trained model going to score 48 on DeepSWE v1.1 ?

https://mimo.xiaomi.com/rl/

imjonse 4 hours ago | parent

That is the RL training cost only. Their announcement blog mentions this: https://mimo.xiaomi.com/mimo-v2-6#scaling-rl-fully-open-sour...

drbscl 4 hours ago | parent

My understanding of tech salaries in China is that they are pretty decent, but not as high as in SF; closer to typical European salaries.

Mostly due to lower cost of living; Shenzhen is way cheaper than SV

f6v 3 hours ago | parent

I seriously doubt salaries are included. It must be just the electricity and GPU costs.

drbscl 5 minutes ago | parent

In these metrics, yes. In the reported training budgets of anthropic/openai, who knows?

tensegrist 3 hours ago | parent

where's the flash model? it's out already isn't it

segmondy 2 hours ago | parent

Mimo2.5 is really good, but tended to loop too much for my taste. Locally, Pro2.5 wasn't much better. I would reach for it for one shots, hopefully they sorted it out with v2.6, it's a model that's slept on by many. I found that most people that used it did so because it was free. It's a top model worth exploring if you have never given it a go.

deanc 1 hour ago | parent

Why is it labelled as #1/114 for artificial intelligence (at the top of the page) but if you scroll down and look at the graphs it's obviously not?

MallocVoidstar 1 hour ago | parent

Hover the #1/114 and it shows only open weights, that's probably why

deanc 1 hour ago | parent

Feels a little pointless and disingenuous to present it this way (as default) unless you specifically filter it as such