New Techniques to Improve Small I/O Workloads in Distributed File Systems
summary
Introduces a practical layer that runs on Ceph object storage and a DeltaFS fork that containerizes the academic DeltaFS stack, using an LSM-backed metadata layout, client-side libDeltaFS integration, and environment flags DELTAFS_INPUT/DELTAFS_OUTPUT so containers publish snapshots on exit. Claims: small I/O latency drops by removing per-path round trips via batched creates and snapshot publishing; works well for cooperative analytics clients but requires new snapshot import/export workflow and merging remains a proof‑of‑concept for conflicting writers.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 2:04Use Case
- 3:39Small Transfer Problem
- 5:01Metadata
- 5:58Clientside Cache
- 6:30corpus of work
- 6:56tableFS
- 7:23TableFS Contributing Factors
- 8:50IndexFS
- 10:48BatchFS
- 12:05Snapshots
- 12:38Renaming
- 13:25Merge
- 15:14Demo
- 17:01Conclusion
related talks
34mModernizing Ceph Deployments at CERN: CephFS and Object Storage Across Data Centres
49mIntro to Ceph, the Distributed Storage System
17mSingularFS: A Billion-Scale Distributed File System Using a Single Metadata Server
35mMaking Ceph Fast in the Face of Failure
28mAsynchronous Directory Operations in CephFS
15mInfiniFS: An Efficient Metadata Service for Large-Scale Distributed Filesystems
02h 59mCeph Storage: a Caffeinated Primer
29mGlobal Deduplication for Ceph