Sharding: from scratch to Yandex Disk
summary
A detailed look at how Yandex.Disk’s database sharding evolved: the talk covers the architecture changes, the operational and design problems encountered, and the concrete solutions applied to scale and manage the sharded storage system.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:22Why sharding
- 5:00Sharding key choice
- 8:20MongoDB issues and switch
- 11:17Manual sharding on PostgreSQL
- 15:58Replicas and storage units
- 19:33Shard load limits
- 23:27Data migration between shards
- 29:51Sharding evolution in Yandex Disk
- 31:15Q&A
related talks
39mMailing Adventures with PostgreSQL: Taming 650+ Shards and Surviving
50mHow PG Sharding Works in Yandex Taxi’s Processing
51mYandex Database: Distributed Queries in the Clouds
40mDatabases in the Cloud
50mHow We Built a Distributed Queue Service at Yandex
23mSharding for hyperscale and geographically distributed relational databases
52mTo Shard or Not to Shard
40mStorage Architecture for Yandex.Direct Advertising Objects