A re-introduction to s3fs
summary
s3fs mounts S3 via FUSE and is useful when POSIX access to object storage is required but not when low-latency small updates or huge directory listings matter. It exposes close-to-open consistency, caches metadata for ~15 minutes, and suffers random-write amplification (small writes force multi‑MB multipart uploads). Directory listings use LIST+HEAD per object so tens of thousands are OK, millions are not. Tune parallel_count, multipart_size and use_cache; recent work adds server-side copy for partial updates and flushes ~5GB to avoid rewrites.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:37Overview
- 1:47Usage
- 2:23Compatibility
- 4:08Drawbacks
- 5:52Close to Open Data Consistency
- 6:53S3 Implementation Consistency
- 8:16S3fs Performance
- 10:44S3fs Updates
- 12:16S3fs Pitfalls
- 13:19S3fs vs NFS
- 14:17Alternatives
- 15:01Summary
- 15:30Outro
related talks
16mA re-introduction to s3fs
26mImprove FUSE Filesystem Performance and Reliability
32mBringing up FUSE mounts C/R support
36mFlash Friendly File System
17mCoFS: A Filesystem for Fast Container Startup
24mFuse mounts recovery and Checkpoint/Restore
09mFour fake filesystems!
30mTo the Cloud and Beyond, Accessing Files Remotely from Linux via SMB3.1.1