Cost-efficient Archive Cloud Storage with Tape: Design and Deployment
summary
TapeOBS claims tape reduces TCO 4.95× and CO2e by 97% and runs in production storing hundreds of petabytes since 2024. It uses a 4% HDD staging pool to make a fully asynchronous tape pool, handles 80s tape switches and a 4:1000 drive-to-tape ratio by statically splitting drives into dedicated drives groups, and lowers restore fanout with batch erasure coding. Deployments show 14 racks per pool (≈140 PB at EC12+2) and steady background flush/GC throughput.
AI-written summary · about
chapters (11)
- 0:00Tape OBS archive storage
- 1:24Tape versus HDD for archives
- 2:25Tape library fundamentals
- 3:09Tape OBS architecture
- 4:14Design principles for tape storage
- 5:17Asynchronous tape pool with HDD staging
- 6:44Minimizing drive thrashing
- 7:47Batch erasure coding for restores
- 8:30Local storage engine for tapes
- 10:38Production deployment insights
- 12:46Conclusion and future
related talks
41mMaximising Data Storage Efficiency with Ceph as a Caching Layer for Tape Archiving
17mBoosting Cost Efficiency by Cost Aware Architecture
10mArchiving and digitally preserving your data with AWS
23mHow Atlassian is migrating 40PB data with 99.99% reliability
45mUber achieves millions $ of savings managing disk IO across HDFS cluster
31mFinding the Needle in the Haystack: Predicting Storage Device Failures in...
41mFinOps, how data engineers get their cloud cost under control
57mDriving innovation and insight with storage