Debuggers and eBPF: Bringing Debugging to Production
summary
A Delve rewrite using eBPF replaces slow ptrace stops with in-kernel probes to trace Go functions with microsecond overhead instead of seconds, using libbpfgo and uprobes to attach uprobe/uretprobe trampolines, embedding the eBPF object in the binary, and exchanging per-address metadata via ring buffers and maps. It handles Go ABI changes, passes argument layouts and goroutine id offsets from userland, temporarily removes uretprobes during stack copies to avoid panics, and requires privileged load rights or CAP_SYS_ADMIN and optional external debug info.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 0:22Overview
- 1:53Derek Parker
- 2:32Why
- 2:58PTrace
- 3:48Testing
- 5:11Why is ptrace so slow
- 6:12Solution eBPF
- 6:26Benchmarks
- 6:38Why is eBPF so fast
- 7:18eBPF requirements
- 8:56Embedded eBPF
- 9:44Lowlevel implementation details
- 10:21eBPF information
- 12:52eBPF probes
- 13:30Updating eBPF map
- 15:36Getting data back from eBPF
- 16:46Upsides and downsides
- 17:01eBPF programs
- 18:31Fighting the verifier
- 19:00Small stack requirements
- 19:16Rep probes
related talks
46mReal World Debugging with eBPF
54mTracing Go Programs with eBPF!
25mKernel tracing with BPF
27mLow-Overhead Tracing Using eBPF for Observability into Kubernetes Apps and Services
39meBPF Superpowers for Go programmers
37meBPF: Revolutionizing Observability and Telemetry for Modern Applications
33mBuilding a Go Profiler Using Go
27mBeyond printf & tcpdump: Debugging Kubernetes Networking with eBPF