How We’re Scaling Discovery at Netflix Reliably
summary
Mosaic constructs Netflix member pages for ~300 million households and >300 client surfaces and faces a combinatorial explosion of server driven configurations. The team builds four end‑to‑end strategies: replay tests sample and replay production GraphQL requests to catch long‑tail library misuses; shadow canaries replay live requests into baseline and canary clusters via Chap and diff responses to surface JVM leaks and emergent regressions without touching users; failure injection testing intercepts gRPC calls using the dependency graph to validate graceful degradation; Mosaic override service lets clients dictate UI for deterministic client tests, and these approaches materially reduce regressions and unsafe rollouts.
AI-written summary · about
chapters (8)
- 0:00Netflix scaling challenges
- 2:29Mosaic in member page
- 7:05Testing Mosaic complexity
- 8:22Replay tests and shadow canaries
- 10:37Failure injection for resilience
- 13:32Client UI testing
- 16:01Mosaic override service
- 17:41Key takeaways
related talks
19mKeynote: Making and Keeping Netflix Highly Available
47mCloud Security at Scale and What it Means for Your Application
27mBehind the Scenes: Running Critical Services @ Netflix
38mChaos Engineering: A Step Towards Resilience
55mIncident Management at Netflix Velocity
01h 52mDeep Dive into the Cloud Native Open Source with NetflixOSS
44mTechniques Netflix Uses to Weather Significant Demand Shifts
41mHow Netflix Embraces Failure to Improve Resilience and Maximize Availability