A “Moving SLO” For Machine Learning and AI Services
summary
Proposal reframes SLOs as residual error as SLO by running a predictor corrector (Kalman‑family) over model inputs and observable outputs and averaging the error per user and per funnel step. Technique uses shadow baseline models and filter banks (constrained, extended, unscented variants) to detect drift early, enable online model swapping, and estimate unmeasurable states (fraud risk, rocket plume temperature). Approach is theoretical and is not built end to end though Kalman filters are proven in GPS phones cars and avionics.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:11Service level objectives
- 7:05Traditional SLO limits for ML
- 10:13Unmeasurable states problem
- 16:15Reality in ML happens once
- 19:04Model drift and customer behavior
- 22:59Kalman filters for ML monitoring
- 30:06SLOs with predictor-corrector models
- 35:44Real-world ML applications
- 37:32Q&A
related talks
23mGoing from 30 to 30 Million SLOs
19mImproving Reliability @ Meta: By Analyzing Historical Events That Led to SLO Violations
31mServerless Observability: a case study of SLOs
30mTechniques for SLOs and Error Budgets at Scale
09mHow to answer tricky questions the SRE way
29mMeasuring Reliability: What Got Us Here Won't Get Us There
30mMetrics for good, not evil
40mScaling SLOs with Kubernetes and Cloud Native Observability