Think Outside the VM: Unobtrusive Performance Measurement
summary
When VM tracing is too risky, measurement moves outside the VM and uses Linux kernel facilities to sample native state and reconstruct Erlang stacks: perf (perf record/top), perf map plus DWARF unwinding, ELF/elfutils hacks, and reading target memory with process_vm_readv. A released tool called rolling-sample intermixes native and Erlang frames, reveals NIF and GC hotspots (timer_now_diff surprisingly hot), estimates skid, and supports blame-mode and slow‑down experiments. Approach works at tens of thousands RPS but tools are brittle and experimental.
AI-written summary · about
chapters (7)
- 0:00The challenges of profiling
- 3:35Analyzing system interfaces
- 9:38Leveraging perf for analysis
- 14:04Visualization and metrics
- 19:32Hacking custom stack traces
- 34:09Practical workflow tactics
- 38:41Future directions and Q&A
related talks
38mAdvanced BPF profiling techniques: how to escape the curse of NMI and fetch ELF...
37mAdvanced Performance Profiling Using Perf Tools
44mObservability Tools C++: Beyond GDB and printf - Tools to Understand the Behavior of Your Program
34mChallenges in Measuring Erlang Application Performance
29mPerf Support in Xen
51mLinux Systems Performance - Architecture
44mJVM Profiling under the Hood
45mPerf tools updates and beyond