Tracing Kafka with OpenTelemetry
summary
Demonstrates that Kafka can be traced end to end using OpenTelemetry and an OTLP endpoint with a Java Spring Boot sample app, a Java agent for auto-instrumentation, and Kafka producer/consumer interceptors that add trace headers. Auto-instrumentation produces traces but leaves uninstrumented time gaps; manual spans fill those gaps. Visualized in New Relic or Jaeger the trace shows a big consumer lag gap. Notes a 70% adoption prediction for OpenTelemetry by 2025 and reports a reported ~10% memory increase with some auto agents.
AI-written summary · about
chapters (14)
- 0:00Tracing Kafka with OpenTelemetry
- 3:12Why tracing Kafka matters
- 6:06OpenTelemetry overview
- 10:43Designing OpenTelemetry for Kafka
- 14:59Auto instrumentation demo
- 20:57Manual instrumentation demo
- 27:14Auto vs manual trade-offs
- 31:27Sampling and rate limiting
- 33:33Metrics vs reliability
- 34:58OpenTelemetry in Kubernetes
- 38:07Memory overhead of auto-instrumentation
- 40:14Monitoring Kubernetes API
- 41:16Alerts for slow requests
- 42:40Q&A
related talks
41mTracing event-based systems with OpenTelemetry
30mPractical introduction to OpenTelemetry Tracing
38mPractical introduction to OpenTelemetry tracing
47mI'm Tracing You!
50mI'm Tracing you!
45mPractical introduction to OpenTelemetry tracing
30mInstrument Your Java Applications with OpenTelemetry!
43mPractical Introduction to OpenTelemetry Tracing