What Timescale's Approach to Dropping Chunks Tells Us About Concurrent DDL
summary
Timescale implements chunk drops by removing entries from its internal catalog and using custom planner hooks so it takes a lighter lock on the parent (share update) and an access exclusive on the child, letting drops happen without blocking long reads. That works for many analytics workloads but allows theoretical serializable snapshot anomalies. Proposed core-safe changes include an explicit waiting state for DDL and the ability to hand off to a background worker to finalize when the XID horizon advances, plus rethinking create index concurrently and monitoring views.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:28What is concurrent DDL
- 5:26Partition management pain points
- 9:41Detached concurrently explained
- 14:13Timescale's chunk dropping approach
- 18:57Downsides of Timescale's method
- 23:36Why Timescale's trade-offs work
- 25:44Real vs fake problems in DDL
- 28:16Proposed path forward
- 33:35Need for general solutions
- 35:00Q&A session
related talks
31mHow to Avoid Pitfalls in Schema Upgrade with Percona XtraDB Cluster
39mTimescaleDB: Re-engineering PostgreSQL as a time-series database
24mTSDB Data Compactions on Scale Without Headache: Now & Future
20mChange Data Capture in Distributed Systems
17mScheduled Deletions At Scale
01h 00mDistributed transactions implementation trade-offs
30mBuilding a Time-Series Database for Point-in-Time Lookups
48mLightning Talks