The Evolution and Innovation of CubeFS
Talkat KubeCon + CloudNativeCon North America 2022 · on Friday, 2022-10-28· at 11:00· Virtual Platform Only·
summary
CubeFS is a cloud-native distributed file system compatible with S3/POSIX/HDFS, provides replication and erasure coding and LRC (reducing redundancy from ~3x to ~1.3x for multi‑AZ), and ships version 3 with erasure coding plus 3.1 adding hybrid‑cloud cache and QoS flow control. Deployed at Oppo for AI platforms and Spark remote‑shuffle, it handles millions of small files, creates snapshots in seconds, and reports benchmark speedups of ~20–300% on public‑cloud GPU workers while cutting cross‑region latency to microsecond levels.
AI-written summary · about
chapters (14)
- 0:00Start
- 1:38Community Growth
- 2:06Background
- 2:30CubeFS-Elasticity and scalability for metadata
- 3:58CubeFS-Optimized for small files
- 4:37CubeFS-performance comparison
- 6:33User Adoptions - OPPO
- 10:27DataLake Solutions: Multi-protocal capability
- 10:59DataLake Solutions: Erasure Code Subsystem
- 16:29Hybrid Cloud Acceration
- 23:05DataLake Solutions: QOS
- 24:30Al Platform solution:snapshot
- 27:58Remote shuffle solutions
- 31:43RoadMap
related talks
29mCubeFS in Action: Empowering Users Through Case Studies
31mScaling Autonomous-Driving R&D: A 100 PB, 100 Billion-File Open-Source Elastic File...
42mProduction FS: Adapt or Die
14mFalconFS: Distributed File System for Large-Scale Deep Learning Pipeline
47mDesign Decisions Behind Cube Store - a Blazingly Fast Open-Source Rollup Storage Layer
29mAccelerate your cloud architectures with the right storage options
19mFlatFS: Flatten Hierarchical File System Namespace on Non-volatile Memories
43mCeph and Flash