Metadata for the Cluster: A Key-Value Heroes Race
summary
Compares key-value databases as candidates for storing metadata in a high-performance storage cluster, examining whether popular key-value systems can handle the required load (millions of ops/sec with sub-millisecond latency). Discusses real-world evaluation and trade-offs when using key-value stores as metadata backends for a high-throughput cluster.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:41What are cluster metadata
- 3:46Ceph issues and storage market
- 8:22Metadata requirements
- 10:33How block storage works
- 11:55Metadata volume in cluster
- 13:06Metadata storage options
- 16:57Testing embedded databases
- 21:18Testing methodology
- 22:45Reliable database writes
- 24:08Database test results
- 33:50Testing dedicated databases
- 37:48Aerospike results
- 40:17Nuances and improvements
- 43:49Q&A
related talks
10mThe unreasonable effectiveness of key value databases
32mThe Future of Metadata in Prometheus: Enhancing Storage and Usability
39mBig Data Systems Performance: The Little Shop of Horrors
05mInterview
59mKey-Value Stores in Large Projects
30mStateful Superpowers: Explore High Performa...
19mBuilding KVStore for ML Workloads
29mMetadata Systems for End-to-End Data & Machine Learning
