Advanced Observability Strategies for Media Workflows at Netflix
summary
At Netflix scale a single episode produces 1 million spans per episode, 27,000 microservice calls, 140 video encodes and 552 audio encodes consuming ~122,000 CPU hours, so observability shifts from traces to streaming. The team builds a Kafka + Flink pipeline that ingests spans, metrics and metadata, pre-aggregates top queries into Elasticsearch and Iceberg in near real time, enforces request ID adjacency to collapse fan‑out, and operates with no sampling of traces to enable cost, SLO and long‑pole latency analysis.
AI-written summary · about
chapters (9)
- 0:00Media workflow observability intro
- 7:33Observability challenges evolution
- 14:23Monolith to microservices
- 22:11Scaling with OpenTelemetry
- 33:15Request IDs for debugging
- 40:24Custom visualization at scale
- 48:41Real-time business insights
- 53:20Metrics, traces, metadata
- 56:11Future directions
related talks
31mThe Observability Data Lake, 1 year on
31mMastering Near-Real-Time Telemetry and Big Data: Invaluable Superpowers for Ordinary SREs
50mIntuit Journey To Unified Observability at Scale: Challenges, Be...
38mIt's Simple…
12mA Principled Approach to Monitoring Streaming Data Infrastructure at Scale
26mUnified End-to-End Observability: How Comcast Generates SpanMetrics at Enterp...
31mFrom Alerts to Insights: Performing Trace-Based Causation at Scale
12mHarnessing Chaos Engineering to Build Robust Systems