LoopDelta: Embedding Locality-aware Opportunistic Delta Compression in Inline...
summary
LoopDelta embeds delta compression into inline deduplication and implements four concrete techniques: dual-locality-based similarity tracking, locality-aware prefetching, cache-aware filter, and inversed delta compression. It detects many more similar chunks by combining logical and physical locality, prefetches base chunks to avoid extra I/Os on write, filters to avoid extra reads on restore, and rewrites infrequently reused containers to enable inverse deltas. Evaluation on RDB, web, source and synthetic datasets shows 1.24–10.97× higher compression over dedup, 1.2–3.57× faster restores, and no notable backup throughput loss.
AI-written summary · about
chapters (19)
- 0:00LoopDelta research intro
- 0:44Data reduction in backups
- 2:10Redundancy locality in dedup
- 3:35Sketch indexing overview
- 4:16Logic locality indexing limits
- 5:24Physical locality indexing limits
- 7:32Full indexing advantages
- 8:15Dual locality sketch indexing
- 8:57Backup throughput challenges
- 9:59Restore performance issues
- 12:49Inverse compression concept
- 14:13Inverse compression example
- 15:39LoopDelta solution overview
- 16:21Performance datasets
- 17:24Chunk detection effectiveness
- 18:29Restore performance gains
- 18:51Compression ratio analysis
- 19:56Comparison with approaches
- 21:01Backup throughput results
related talks
16mThe Evolution of Delta Lake
19mFinesse: Fine-Grained Feature Locality based Fast Resemblance Detection...
46mOpen Table Formats in the Wild: From Parquet to Delta Lake and Back
40mDelta for your Data: How to efficiently track and store deeply nested data changes
20mDelta Lake: Turn your Data Lake into a Lakehouse
29mSliding Look-Back Window Assisted Data Chunk Rewriting for Improving Deduplication...
39mDelta Lake: Diving into Data Lakes Without the Downsides
32mDesigning Data Infrastructure in the Age of Generative AI