A Homegrown S3 Server: Challenges of Building Stable Storage
summary
Describes Ozon's migration of object storage to their in-house S3-compatible server Lusca, covering choices of index storage technologies, techniques used to implement strong consistency on top of an eventually-consistent backend, and insights about ScyllaDB background processes plus custom algorithms developed to ensure a stable storage system.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 2:35Rados Gateway issues
- 6:03Building custom gateway
- 10:47ScyllaDB transaction problems
- 15:04Object listing and partitioning
- 21:24Performance spikes and large partitions
- 27:45Halloween zombies and Log Batch
- 33:34Versioning and data recovery
- 35:16Summary
- 35:59Q&A
related talks
32mElevating Scalable Object Storage: A Comprehensive Exploration of Ozone’s Trailblazing Capabilities
54mOK S3: Building the System Ourselves
31mThe Design of a Modern Distributed Object Storage
42mApache Ozone - State of the Union
44mOpenSearch’s Quest for Disaggregated Storage: The Strangler Fig Patter...
44mInside S3
36mStorage Wars
29mAccelerate your cloud architectures with the right storage options