ParaRC: Embracing Sub-Packetization for Repair Parallelization in MSR-Coded Storage
summary
ParaRC leverages sub-packetization of MSR codes to parallelize sub-block repairs, models the problem as a colored graph called a PEC DAG to expose the trade-off between repair bandwidth and maximum per-node repair load, and uses a heuristic to precompute an approximate MLP (min-max-load point) offline. Prototype on HDFS and experiments on Alibaba Cloud show reduced degraded-read and full-node-recovery times by up to 59.3% and 39.2% respectively; precomputation for n=16, k=12 takes about two days.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:36Repair Penalty of RS Codes
- 2:39Reducing Repair Bandwidth
- 3:44Clay Codes
- 4:11Reducing Maximum Repair Loa
- 5:12Repair Parallelization for MSR Co
- 6:07Motivating Example
- 7:00Summary
- 7:24Our Contributions
- 9:26Traffic Table
- 10:00Trade-off Analysis
- 11:03Heuristic
- 14:07Finding Approximate MLP
- 14:59ParaRC Performance on Alibaba
- 15:25Conclusions
related talks
16mBoosting Full-Node Repair in Erasure-Coded Storage
11mExploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage
15mLESS is More for I/O-Efficient Repairs in Erasure-Coded Storage
10mRevisiting Network Coding for Warm Blob Storage
18mDRBoost: Boosting Degraded Read Performance in MSR-Coded Storage Clusters
15mSelectiveEC: Selective Reconstruction in Erasure-coded Storage Systems
29mAccelerating Filesystem Checking and Repair with pFSCK
51mConvertible codes: Adaptive coding for large-scale data storage