Auto-Instrumentation for GPU Performance using eBPF
summary
eBPF hooks into CUDA APIs to record kernel launches, grid sizes and host↔device memory copies and export them as Prometheus metrics tied back to Kubernetes pods via OpenTelemetry; enabling live, low‑overhead profiling without rebuilding. Measured results show concrete wins: P95 48ms → 11ms (4×) after applying Mirage, kernel launches ↓100× and grid size rising from ~800 to ~3000 with device→device copies dominating, and on an A100 a different model yields kernel launches ↓10× and ~2× latency improvement.
AI-written summary · about
chapters (12)
- 0:00GPU auto-instrumentation intro
- 1:22Why monitor GPUs now
- 4:08Current GPU tools limits
- 7:39GPU monitoring challenges
- 10:08What is eBPF for GPUs
- 11:12OpenTelemetry eBPF for CUDA
- 13:20Demo: kernel & memory metrics
- 14:24Kernel data for insights
- 15:27Mirage optimization case
- 17:13Thunder Kittens case
- 19:20GPU efficiency metrics
- 19:42GPU monitoring future
related talks
28mAuto-instrumentation for GPU Performance Using eBPF
29mAuto-instrumentation for GPU performance using eBPF
42meBPF-Powered Observab...
33mAuto-instrumentation of Prometheus for RED Monitorin...
24mMaking the Kernel Speak Kubernetes: Unlocking eBPF’s Power for Observability
31mZero instrumentation observability with eBPF
42mCommunity Driven Breakout Session: eBPF and Auto-Instrumentation
35mSidecarless Mesh Monitoring with eBPF and Grafana