Market Data: Applying SRE Techniques to Legacy Designs
summary
Market data platform handles up to 700 billion ticks/day with a 5 ms latency target; initial wins come from Black box probes using the internal Market Data Toolkit and Prometheus to reveal a weekend outage and a thread leak and to quantify cross‑DC RTT impact. Moving a ticker plant into AWS exposes TCP/multicast and SLA limits so the architecture shifts to per‑feed access points, increasing failure domains and isolation. White‑box work uses exporters and OpenMetrics, plus live software‑version metrics to drive fixes.
AI-written summary · about
chapters (5)
- 0:00Introduction
- 2:54Market Data Engineering
- 5:36Blackbox Monitoring
- 9:28Architecture Change
- 18:30Conclusion
related talks
41mTracing Real-Time Distributed Systems
01h 06mPatterns for scalability and availability in (trading) systems
26mDistributed Tracing: All the Warning Signs Were Out There!!!
37mReactive Architecture on AWS: from 0 to 1.6M events/minute
47mProduction Engineering When Trading Billions of Dollars a Day
11mMitigating Against Large Scale Systemic Failures in E-Trading
01h 00mSystems @Scale Summer Q&A Week 4
34mDesigning, Building and Testing Deterministic HFT Systems