The what, The from, and The to: The Migration Games in Deduplicated Systems
summary
Frames deduplicated migration as an optimization to minimize total stored blocks subject to per-volume load balance and a traffic budget, and evaluates three planners: a fast greedy that runs in seconds and yields modest space savings, a clustering method that groups files by Jaccard similarity and location and typically matches ILP quality, and an ILP that targets optimal size but often hits a six-hour timeout. Experiments on Microsoft workstation, Stony Brook backups and Linux kernel sets show clustering ≈ ILP, greedy much worse, and load‑balancing can cut achievable deletion percentage substantially.
AI-written summary · about
chapters (9)
- 0:00Intro
- 1:52File migration in deduplicated systems - goals
- 3:14Greedy • Iterative remapping of files to volumes
- 4:33Hierarchal clustering Clustering group similar files • Assign each cluster to a volume
- 6:03Approach 3: Integer linear programming (ILP) - Directly search for theoretically optimal plan
- 7:24Experimental Datasets
- 8:38Reduction in system size
- 10:18Runtime
- 11:49Conclusions
related talks
38mEfficient Reliable Database Migration: A Legacy Innovation Story
12mThe Dilemma between Deduplication and Locality: Can Both be Achieved?
14mRe-think Data Management Software Design Upon the Arrival of Storage Hardware with
17mLight-Dedup: A Light-weight Inline Deduplication Framework for Non-Volatile...
45mIt's Not You, It's Me: Breaking Up with Massive Tables via Partitioning
53mData Deduplication For Redundant Graph Structures Files
18mPreview: KV-Stores and Databases
45mData Placement in Linux: Evolving Block and File I/O