Stop Performance Regressions Before Release Valkey’s CI Benchmarking
summary
An open CI benchmarking pipeline called Valkyrie perf runs on bare metal C6g instances with CPU/process pinning, IRQ affinity and IRQ balancing disabled to achieve determinism; results feed into Postgres and visualized in Grafana+Postgres dashboards. Determinism yields a 15% higher set throughput in one experiment and reduces run-to-run coefficient of variation from about 33% to ~1.1%, letting workflows (GitHub Actions on self‑hosted runners) bisect commits, label PRs for on‑demand 7‑hour benchmarks, and auto-detect regressions before release.
AI-written summary · about
chapters (5)
- 0:00Preventing regressions
- 2:55Deterministic benchmarking
- 7:54Grafana benchmark results
- 13:06PR benchmarking workflow
- 16:19Q&A
related talks
36mAutomated Performance Testing of Postgres
16mServiceLab: Preventing Tiny Performance Regressions at Hyperscale through...
20mQemu Performance Regression CI
24mgRPC Performance and Testing: A Maintainer Perspective
36mFighting Regressions with Benchmarks in CI
01h 00mContinuous Regression Testing for Safer and Faster Refactoring in C++
41mThe Importance of Continuous Regression for HW & SW Development
33mPerformance Testing in Agile and DevOps