Discard-Based Garbage Collection for Distributed Log-Structured Storage Systems in...
summary
ByteStore implements a new Discard-and-Compaction GC called DisCoGC that reclaims large contiguous stale ranges without rewriting valid data, implemented by scanning block metadata, issuing batched discard requests into UFS, merging trims, and falling back to low-frequency compaction to defragment. Production deployment shows 32% lower logical write amplification, 10% lower space amplification and an overall 20% reduction in TCO with near-identical average latency and modest increase in physical writes.
AI-written summary · about
chapters (12)
- 0:00Disco GC introduction
- 2:29Traditional GC limitations
- 3:54Workload patterns from traces
- 5:00Discard GC mechanism
- 6:28Multi-layered storage challenges
- 8:57Metadata updates and contention
- 10:23Balancing discard and compaction
- 11:09Optimizing SSD trim operations
- 12:12Evaluation setup and workloads
- 13:17Performance results and TCO
- 14:25Lessons learned and recommendations
- 15:51Conclusion
related talks
52mThe Computer Science behind a modern distributed data store
45mOK Log: Distributed and Coördination-Free Logging
35mSystems @Scale Summer Q&A Week 3
19mBuilding a Performant Configuration-Store Layer
21mNew Techniques to Improve Small I/O Workloads in Distributed File Systems
17mDuhu: Shared Disaggregated Memory for Distributed Data Processing Frameworks
34mDistributed systems 102
19mScalog: Seamless Reconfiguration and Total Order in a Scalable Shared Log
