Strimzi & OpenTelemetry: Building End-to-End Observability for Kafka on Kubernetes
summary
Strimzi plus OpenTelemetry shows end-to-end observability for Kafka on Kubernetes by wiring producer→broker→consumer traces into backends like Grafana, Tempo and Jaeger, using either an external agent (Java agent/sidecar auto-instrumentation) or direct SDK instrumentation in Java, Go, Python, etc. Prometheus metrics from Strimzi combine with traces to pinpoint drops and latencies even at extreme scale (claims of ~2 billion messages per second). The talk demonstrates importable Grafana dashboards, recommends a centralized collector, and enforces trace context and Kafka headers.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:42Why Kafka Observability Matters
- 2:40Enabling Tracing
- 4:53OpenTelemetry
- 6:43Example
- 7:22OpenTelemetry SDK
- 8:29Kafka Inping
- 11:30Correlation
- 12:04Kafka Internals
- 13:09Kafka Dashboard
- 13:44Key takeaways
- 14:47Best practices
- 16:02Roadmap
- 16:36Resources
- 17:00Summary
- 17:27Questions
related talks
30mFrom Running to Operating: Real-World Strimzi in Production
31mSimplifying Apache Kafka on Kubernetes With Strimzi
42mTransition to Apache Kafka on Kubernetes With Strimzi
26mProduction Grade Kafka on K8s
34mHow to Survive Managing Strimzi in an Enterprise
35mApache Kafka as a Monitoring Data Pipeline
44mTaming Apache Kafka Using GitOps
23mWhen Kafka Goes Cloud Native: Observability That Actually Works!