Sharding Postgres without doing it yourself
summary
A production-ready, open-source stateless Postgres query router for horizontal scaling via sharding is presented. It implements the Postgres protocol, is written in Go, and is intended to enable transparent sharding of PostgreSQL databases without building a custom sharding layer.
AI-written summary · about
chapters (29)
- 0:00Introduction
- 1:52Problem statement
- 3:49Why sharding is necessary
- 5:51SPQR architecture
- 9:45SPQR router
- 12:13Coordinator and balancing
- 13:19Importance of balancing
- 14:28Performance challenges
- 16:35Query parser improvements
- 16:57TPC-C benchmark
- 19:30Live demo
- 23:25Open source
- 24:07Call to action
- 24:53Q&A
- 25:14Consistent hashing
- 28:06Cross-shard transactions
- 30:17Multi-shard requests
- 31:41Comparison with others
- 32:26When to use SPQR
- 33:10Development background
- 33:31SPQR name origin
- 34:13Request cancellation
- 34:58Sharding key config
- 36:24Multi-table rules
- 37:09Stateless router
- 37:29Routing reads/writes
- 38:15Target audience
- 39:27Index and field support
- 40:09Final Q&A
related talks
45mThe Future of Postgres Sharding
45mThe Future of Postgres Sharding
33mFive Sharding Data Models and Which is Right
42mPostgreSQL Distributed & Secure Database Ecosystem Building
34mPostgreSQL sharding, state of the art
53mBreakout Session
47mpg_shardman: Sharding in PostgreSQL
51mScaling Postgres to the next level at OpenAI