Everyone Loves File: File Storage Service (FSS) in Oracle Cloud Infrastructure

on YouTube ↗22m

" summary

Oracle FSS implements a managed NFS by building a pipelined Paxos replicated backend: a scalable block store offering linearizable multipage limited-size transactions (the system supports a 15-page MPSC atomic primitive) and a scalable B-tree that maps keys to pages using self-validating nodes. Snapshots use versioned key-value pairs, presentation servers are stateless, extents use 5-way replication, and a Paxos-replicated hash table bootstraps config. Throughput follows a simple model (bandwidth/5 with queuing losses, practical running ~1/3 peak); unloaded write latency is ~4× a Linux NFS server on NVMe.

AI-written summary · about

" chapters (15)

1. 0:00Intro
2. 0:40What is File Storage Service
3. 1:50Architecture of File Storage Service
4. 4:15Problems with Standard File Servers
5. 4:47Scale Up: FSS Distributes Data Structures Across Servers
6. 7:01Represent Inodes as Tabular Data instead of implementing the inodes directly
7. 8:08Store the Tabular Data in a Distributed B-tree
8. 9:31Updating the B-tree Requires Atomic Operations Across Servers
9. 11:25The Problem with 2PC
10. 12:04Paxos Doesn't Get Stuck
11. 12:48FSS Uses Paxos to Implement two-phase commit
12. 13:18How Big is an Extent?
13. 14:26Multipage Store Conditional (MPSC) On top of 2PC, We program FSS using an
14. 16:46A Simple Throughput Model
15. 17:57Looks Like a Speedup Curve

" related talks

• 17mSingularFS: A Billion-Scale Distributed File System Using a Single Metadata ServerJiwu Shu, Youyou Lu, Shaoxun Zeng, Xiaojian Liao, Hao Guo, Wenhao Lv · USENIX ATC '23
• 16mHadaFS: A File System Bridging the Local and Shared Burst Buffer for ExascaleBin Yang, Wei Xiao, Qi Chen, Weiguo Liu, Wei Xue, Xiaobin He, Jie Gao, Shupeng Shi, Dexun Chen, Zuo-ning Chen · FAST '23: 21st USENIX Conference on File and Storage Technologies
• 15mInfiniFS: An Efficient Metadata Service for Large-Scale Distributed FilesystemsJiwu Shu, Yiming Zhang, Youyou Lu, Wenhao Lv, Peile Duan · FAST '22
• 12mFacebook's Tectonic Filesystem: Efficiency from ExascaleRichard Wareing, Abhinav Sharma, Monika Gangapuram, Satadru Pan, Kestutis Patiejunas, Pratap Singh, Wyatt Lloyd, Shiva Shankar P, Theano Stavrinos, Yunqiao Zhang, Atul Sikaria, Pavel Zakharov, Mike Shuey, Guanglei Cao, Christian Preseau, JR Tipton, Ethan Katz-Bassett · FAST '21
• 20mscoutfs: Large Scale POSIX ArchivingZach Brown · Vault '19
• 21mNew Techniques to Improve Small I/O Workloads in Distributed File SystemsDan Lambright · Vault '19
• 45mA Distributed File System for Secure P2P ApplicationsBrooklyn Zelenka · Strange Loop 2022
• 19mConsolidating Storage Backends at Scale with Tectonic FilesystemSatadru Pan · Systems @Scale Summer 2021