How Human Expertise and AI drive Scalable Forecasting Success at
summary
Albert Heijn runs 1 billion forecasts nightly with 2,500 models in production, using LightGBM, Airflow, Kafka, Azure Data Factory and Databricks (moving to Databricks SQL serverless warehouses) plus a Kotlin Spring Boot layer; the stack drops screen loads from ~10–12s to 2–3s, cuts cloud cost ~30%, and supports real-time alerts so humans adjust ~3% of items at peak events; the program spans three years and reduced food waste by 1.3M kg in 2024, showing the engineering and numbers that actually worked.
AI-written summary · about
chapters (10)
- 0:00Global food waste problem
- 2:52Forecasting for grocery retailers
- 4:17Scaling to 1B daily forecasts
- 7:50Three-year journey to 2500 models
- 10:00When ML forecasts fail
- 13:11Human forecast adjustments
- 18:26Real-time forecast adjustment system
- 21:36Scalability bottlenecks
- 24:49Forecasting platform results
- 26:35AI and human expertise
related talks
39mProductionising Data Science at Scale
30mAI in Retail - Advancing Promo Demand Forecasting
23mMachine Learning with a Deep Dive into Automated Forecasting System
35mBuilding a successful inventory planning solution with Amazon Forecast
28mUse case in demand forecasting
25mWeather Forecasting: An Application of Artificial Intelligence, Cloud, IOT &...
28mDemand Forecasting: Accuracy, Stability, Automation
37mForecasting using machine learning methods