~/talk

Breaking Production to Sleep Better: How and Why We Destroy Our Systems on Purpose

TalkinFrenchat Sunny Tech 2026 · 2026-07-02..07-03· [devops-and-delivery, observability]
links:none on file
links:none on file
slides:none on file

Engineers at OVHcloud describe intentionally breaking production systems via organized "Game Days" to validate recovery procedures and uncover real failure points. They explain how to design realistic failure scenarios and ramp up from manual sabotage to large-scale load tests, illustrating with a stress test of a Grafana Mimir cluster (deployed on VMs), which revealed failures in network bandwidth, disk I/O, and Linux kernel limits rather than the application itself.

AI-written summary · about

  1. 0:00Introduction
  2. 3:07Why resilience proof
  3. 6:15From experiment to culture
  4. 11:49Game Day approach
  5. 17:32Cassis load testing
  6. 19:28Stress test failures
  7. 22:59Overlooked dependencies
  8. 26:56Stopping conditions
  9. 30:14Key findings
  10. 33:15Applicability beyond
  11. 36:35Chaos vs benchmarking
  12. 41:08Communication inspiration
  13. 41:55Smaller chaos experiments
  14. 44:26Q&A
~/search
type to search 6,518 conferences, 143,918 speakers and 192,133 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
← →
on a talk: 5 seconds back / forward
0–9
on a talk: jump to 0%–90% of the recording
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.