Production Engineering When Trading Billions of Dollars a Day
summary
Shows concrete incidents and numbers: Knight Capital 2012 burns $460M in 45 minutes and a 97‑email alert storm, a 2005 Tokyo typo sells 600,000 shares at ¥1 and costs ¥27B, so every message can matter and markets can move millions per minute. Describes concrete defenses: event-based actionable alerts to avoid noisy mails, a connectivity monitor and latency SLOs, fine‑tuned colocated hardware and benchmarks, and a global halting service that lets support stop traffic fast and legally while postmortems remove procedural holes.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:17Night Capital incident lessons
- 8:19Trading systems overview
- 12:34Order entry systems importance
- 15:25Tokyo Stock Exchange 2005 error
- 18:35Bugs versus outages in trading
- 22:25Market volatility and time dependence
- 24:30High-performance trading tuning
- 27:19Trading regulations complexity
- 29:02Monitoring in trading necessity
- 31:27Live trading alerts with events
- 36:46Support workflow and response
- 38:33Incident response example
- 41:04Blameless postmortems
- 42:48Production engineering tools
- 44:53Key takeaways and support
related talks
01h 06mPatterns for scalability and availability in (trading) systems
41mTracing Real-Time Distributed Systems
20mMarket Data: Applying SRE Techniques to Legacy Designs
31mDesigning a mission critical event system
31mAlerting for Distributed Systems—A Tale of Symptoms and Causes, Signals and Noise
39m400 Billion Messages a Day with Kafka
40mChaos Engineering at Scale
37mReactive Architecture on AWS: from 0 to 1.6M events/minute