Running FDB at scale at Wavefront
summary
Wavefront runs 500 FoundationDB instances across 35 clusters and reports memory-engine write rates near 800 KHz and SSD writes around 70 KHz. Operation uses a dual-tier memory + SSD architecture with an FDB coprocessor, NVMe enhanced i/o read cache in front of EBS, NUMA and kernel tuning, and a 2:1 storage to transaction process ratio to handle write volume. Cluster lifecycle and hitless fleet replacements are automated with Terraform, Ansible and a bootstrapping system called landing-party. Trace logs are parsed into telemetry with Wavefront FDB Tailor for alerting and automated recovery.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 5:04Tuning Linux
- 6:20Read caching
- 7:42Instance lifecycle
- 8:28Database configuration
- 9:28Landingparty
- 10:50Fleet replacements
- 14:52Trace logs
- 17:00Event tracing
- 19:33Latency
- 20:47Observability
- 23:00Metrics
related talks
16mFoundationDB and Node.js: Year in Production
34mFoundationDB, the Black Knight
21mManaging FoundationDB at Scale
46mFoundationDB: The Best-Kept Secret of New Distributed Architectures!
18mPerformance Profiling in FoundationDB
46mExtreme infrastructure automation with Wavefront
18mFoundationDB Record Layer: Open Source Structured Storage on FoundationDB
24mHow We Saved 5x Migrating from Aurora to FoundationDB