Plug-and-Play Observability: OTel Auto-Instrumentation for LLMs in OpenSearch
summary
Demonstrates a working pipeline that uses OpenTelemetry auto-instrumentation plus an auto-collector and Data Prepper to send traces, metrics, logs and profiles into OpenSearch dashboards with minimal code changes. A FastAPI LLM app with OpenLIT shows real-time traces, per-request token counts, average response times and error rates, and lets teams correlate span IDs to logs and metrics. Auto-instrumentation works zero‑config for many languages, the dashboard visualizations update live, and profiling pinpoints CPU/GPU hotspots to cut unexpected token cost.
AI-written summary · about
chapters (14)
- 0:00Intro to LLM observability
- 1:06Talk roadmap
- 2:32Why observability is critical for LLMs
- 6:40What is OpenTelemetry
- 8:28OpenTelemetry for LLMs
- 10:34Auto-instrumentation with OTel
- 11:59Why pair OpenTelemetry with OpenSearch
- 13:21Integration architecture
- 15:30Live demo setup
- 17:38Running LLM app & traces
- 20:23Analyzing logs & metrics
- 23:10Use cases & best practices
- 24:35OpenSearch tools
- 25:59Key takeaways
related talks
32mFrom Traces To Action: Auto-Instrumenting LLMs for Observability...
22mFrom Telemetry to Insight: Building AI-Powered Observability Pipelines with OpenSearch
18mWhat did AI just do? Making LLMs Observable with OpenTelemetry
34mHow to Do Microservice Observability with OpenSearch
50mGuide to Observability with OpenTelemetry
18mHow To Supercharge AI/ML Observability With OpenTelemetry and Fluent Bit
20mSeeing Is Believing: OpenTelemetry and OpenSearch in Next-Gen Observability
23mOpenTelemetry Auto Instrumentation for Effortless Observability