YHK - yet another hacker news clone

Timestral: Mistral release time-series foundation models

118 points groar 2 hours ago 21 comments

edot 1 hour ago | parent

Very nice. It’s mind-blowing to me that it used to take me months to build and deploy a forecasting model five years ago. Now in minutes I can use something like this, implemented with help from an LLM, and get apparently quite good results (and fine-tune on my data, it seems).

Speed looks nice too. I want an open weight model with this performance now.

GeoffNN 49 minutes ago | parent

Cofounder of The Forecasting Company here - Check ours! It's open-weights: https://huggingface.co/theforecastingcompany/t0-gamma

dfdydx 41 minutes ago | parent

So is Mistral's model, right? Genuine question because I had been following your company - is there any use case where t0 is better than timestral?

OKRainbowKid 45 minutes ago | parent

Perhaps I'm misunderstanding you, but you should be able to download the models at https://huggingface.co/collections/mistralai/timestral

nelaggy 59 minutes ago | parent

crazy how they make an improvement of 0.03 look like a 5x difference

beernet 53 minutes ago | parent

Meh....time series data is fundamentally different from textual data. I see why a time series "foundation model" seems appealing, especially to the marketing folks at Mistral. Real-world usability is very limited, which is probably the reason why the blog post features an imaginary customer rather than an actual reference.

saithound 50 minutes ago | parent

Agered. These are essentially numerology models.

GeoffNN 50 minutes ago | parent

Cofounder of The Forecasting Company here - disagree on this. The models now work quite well and are in prod at a number of places, including very sophisticated places - quant funds, top retail cos etc

beernet 40 minutes ago | parent

You are telling me quant funds prefer a heavy-weight and slow ANN to a fast, specialized model? Maybe in totally non-critical environments, otherwise it's simply too slow, regardless of performance.

mstkllah 34 minutes ago | parent

Do those places also include companies focused on private markets?

AtHeartEngineer 31 minutes ago | parent

Time based data is extremely important, and there are tons of techniques out there to train super fast super small models, but they are a pain in the ass to tune. I could see a foundation model that is kind of good at a lot of things, that could be fine tuned or used when you want something more straight forward to use to analyze a bunch of data.

Honestly this might be very good for feature selection to refine a much more specialized model.

ritzaco 39 minutes ago | parent

Any 'vibe check' on these; I don't have benchmarks but I tried out Le Chonk for general coding work that they claim it works really well at and I was very disappointed, it seemed to make all kinds of obvious errors of the kind that I haven't seen from opus, sol, deepseek flash etc in over a year

maelito 23 minutes ago | parent

Just tested Mistral 4 too in Vibe and it's slow. It works well, but is way too slow. Switched back to Mistral's GLM5.3, which is awesome.

Edit : working with GLM5.3 now, it's also slow. Maybe Mistral's under heavy load.

Disclaimer : Not Cofounder of The Forecasting Company here

joefourier 18 minutes ago | parent

My field is primarily audio/vision ML so I don't have any particular domain expertise, but I don't get how you can lump all "time-series" together in one big bucket. My naive understanding is that the way to improve the accuracy is to gain as much joint information as possible. Like:

> Known-future covariates. Some drivers of sensor behaviour are known in advance, such as production schedules, planned shutdowns, weather forecasts. Timestral can condition on them to improve forecast quality, so a scheduled load increase is expected rather than flagged as a surprise.

To me this should be the entire basis of the system, not a condition to improve forecast quality? Like if you're modelling electricity consumption, if all you have is a window of N previous samples, you're going to get completely beaten by a comparatively simple system that knows the time of day, that there's holidays coming up, the weather forecast, etc.

I would also think that these time-series forecasting models would gain from a sophisticated understanding at least the labels of the covariates? A 5M model has no idea what a "bank holiday" is, it just sees an arbitrary class go from 0 to 1, and won't have any useful inductive biases as to what it means or how to link them to other pieces of information.

E.g., wouldn't you predict €/MWh of electricity far more accurately with access to price of natural gas (itself influenced by many geopolitical factors), breakdown in composition of production of each energy source, predictions about roll-out (requiring a sophisticated LLM capable of understanding news and their implications), etc. What's the use in trying to predict it from 2 variables and a narrow window looking back at just 8192 or 32768 samples?

Or perhaps I misunderstand the use-case and this is more for extremely short term movements?

Y_Y 7 minutes ago | parent

Forecaster of the Confounding Company here :)

You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.

Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).