Don’t Call It 'The Forecast': Structured Prediction at Scale
summary
Forecasting at Spotify becomes a structured prediction system: forecasts are versioned objects in a high‑dimensional space with lineage, schema contracts and probabilistic outputs rather than vague "forecasts." Implementation uses BigQuery + Flight + Forecasting Hub for storage, orchestration, UI and lineage, a product called One Model, and model APIs like Next level for scalable, probabilistic, reconciled outputs. The platform runs about 20 models, turns a quarterly consolidation from weeks into a couple of days, and supports forkable scenario versioning and hierarchical reconciliation.
AI-written summary · about
chapters (5)
- 0:00Introduction
- 2:51Overloaded terminology
- 6:23Forking paths in prediction
- 11:18Forecast structure at scale
- 19:45Impact and future
related talks
24mPlatform-First Forecasting: Engineering Scalable ML Systems
40mMulti-Series Forecasting at Scale with StatsForecast
19mForecasting for Everyone - Foundation Models and the Future of Time Series Prediction
29mHow Spotify Does ML At Scale
39mProductionising Data Science at Scale
28mHow Human Expertise and AI drive Scalable Forecasting Success at
29mProbabilistic demand forecasting at scale
39mProphet at scale to tune & forecast time series at Spotify