Five Sharding Data Models and Which is Right
summary
Claims that sharding is the proven way to scale Postgres, with Citus used to turn Postgres into a sharded distributed DB and examples from Instagram Google and Salesforce; recommends hash vs range with practical shard counts like 128 256 shards and explains how to move shards between nodes, use pg_partman with Postgres 10, and expect Postgres 11 improvements. Notes concrete tactics: add tenant_id on every table, use composite keys, consider HyperLogLog/top‑n for distributed aggregates, and promises up to ~100x gains for parallelizable workloads.
AI-written summary · about
chapters (18)
- 0:00Intro
- 1:23What is sharding
- 3:362 Nodes - 32 shards
- 4:39Five models
- 5:40Hash - The steps
- 7:56Create range of hash values
- 9:00Range - The steps
- 10:23Shard by Geography
- 13:56Real world application
- 14:48Sharding by tenant
- 15:45Common use cases
- 16:12Guidelines for multi-tenant sharding
- 21:34Key considerations
- 22:24Map reduce examples
- 23:58When you use a graph database
- 24:11Very different approach
- 30:15Sensor data - inserting
- 31:04Time series tips
related talks
45mThe Future of Postgres Sharding
45mThe Future of Postgres Sharding
28mEfficiently distributing Postgres with Citus – How to choose the right shard key?
36mPostgres at any Scale
25mData-intensive PostgreSQL: Three ways to scale
40mBeginner's Guide to Partitioning vs. Sharding in Postgres
53mBreakout Session
42mBeginner's Guide to Partitioning vs Sharding in Postgres