How we unified feature engineering across data and backend
summary
Monzo describes a system that defines features once and reuses them offline and online using BigQuery, Spark (batch and streaming on EMR), a key-value store and a feature fetcher, built on open source Kron. It enforces point-in-time correctness, supports backfills and snapshot (midnight) or temporal serving, and chooses aggregation-on-read over per-event updates. Examples: count over a six-hour window and max over 14 days, online latency extra ~10–50 ms, and sampled consistency checks between online and offline values.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 3:57Backend vs data team issues
- 8:06Defining features
- 12:18Deploying features online
- 16:50Monitoring consistency
- 18:55Key learnings
- 21:21Next steps at Monzo
related talks
32mA feature Store for Machine Learning
36mDeclarative Feature Engineering
33mEvolution and Scaling of Feature Store at Uber
30mHow Feature Stores Enable Operational ML
31mBuilding an open-source ML feature store with Apache Flink
45mBuilding a Feature Platform to Scale Machine Learning
33mFeathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature Store
34mSolving the ML Pain You Forgot to Mention