A re-introduction to s3fs

on YouTube ↗16m

" summary

s3fs is a FUSE filesystem exposing S3 objects with POSIX semantics but with clear limits: no hard links or atomic renames, costly list/stat because each entry needs a HEAD (multi-rec max default ~20), and random writes amplify to 5MB due to multipart upload rules. It suits sequential reads/writes of bulky files (typical sweet spot 5MB–5GB, supports up to 5TB) and works across many S3 implementations, uses local /tmp caching (flush every ~5GB), recently fixes concurrency/corruption bugs, and often costs far less than NFS-backed cloud storage.

AI-written summary · about

" chapters (14)

1. 0:00Intro
2. 0:37Overview
3. 1:47Usage
4. 2:23Compatibility
5. 4:08Drawbacks
6. 5:52Close to Open Data Consistency
7. 6:53S3 Implementation Consistency
8. 8:16S3fs Performance
9. 10:44S3fs Updates
10. 12:16S3fs Pitfalls
11. 13:19S3fs vs NFS
12. 14:17Alternatives
13. 15:01Summary
14. 15:30Outro

" related talks

• 16mA re-introduction to s3fsAndrew Gaul · linux.conf.au 2021
• 36mFlash Friendly File SystemJoo-Young Hwang · Embedded Linux Conference 2013
• 26mImprove FUSE Filesystem Performance and ReliabilityTao Peng · KubeCon + CloudNativeCon + Open Source Summit China 2021
• 19mExtension Framework for File Systems in User spaceAshish Bijlani, Umakishore Ramachandran · USENIX ATC '19
• 17mCoFS: A Filesystem for Fast Container StartupYang Yang, Haoze Wu, Li Wang, Jinxu Du, Qingbo Wu, Tao Liu · FAST '26
• 32mBringing up FUSE mounts C/R supportAlexander Mikhalitsyn · Linux Plumbers Conference 2022
• 16mSquirrelFS: using the Rust compiler to check file-system crash consistencyJames Bornholt, Vijay Chidambaram, Hayley LeBlanc, Nathan Taylor · 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24)
• 30mTo the Cloud and Beyond, Accessing Files Remotely from Linux via SMB3.1.1Namjae Jeon · linux.conf.au 2022