SelectiveEC: Selective Reconstruction in Erasure-coded Storage Systems
Talkat 12th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage '20) · 2020-07-13..07-14·
summary
SelectiveEC is a balanced scheduling module that picks which erasure-coded stripes to rebuild and which source and replacement nodes to use, builds bipartite graphs and runs max flow for source selection and perfect matching for replacements, and targets the network-heavy stage that accounts for ~85.23% of reconstruction time. A simulator prototype shows parallelism gains up to 106% and average DRP improvement >97% (97.6% reported); occasional last-batch hotspots affect 5–10% of runs.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:29Distributed Storage Systems (DSSes)
- 1:23Erasure Coding (EC)
- 3:21Breakdown of EC Reconstruction Time
- 3:49Random Data Layout
- 5:36Selective EC
- 6:01Graph Model
- 7:00Select k Source Nodes Dynamically
- 7:45Schedule Reconstruction Tasks Out of Order
- 8:58Select Replacement Nodes Dynamically
- 9:55Evaluation
- 11:10The First Batch
- 12:04Full Batches
- 12:48Summary
related talks
11mExploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage
01h 14mErasure Coding at Scale
16mBoosting Full-Node Repair in Erasure-Coded Storage
15mLESS is More for I/O-Efficient Repairs in Erasure-Coded Storage
17mParaRC: Embracing Sub-Packetization for Repair Parallelization in MSR-Coded Storage
11mStripeless Data Placement for Erasure-Coded In-Memory Storage
15mECROs: Building Global Scale Systems from Sequential Code
15mC2DN: How to Harness Erasure Codes at the Edge for Efficient Content Delivery