A journey toward the columnar data store - lessons from past discussions and new proposals
summary
VCI is an in-memory columnar index that stores read-optimized columns (ROS) and a tiny write-optimized store (WOS) with on-the-fly conversion to temporary local ROS so scans include recent updates. It’s implemented as an index AM + custom scan + hooks and claims 4.4–46× faster scans in reported TPC/DBT3 measurements while showing almost no OLTP update slowdown. Prototype exists as a patch on the community thread; current gaps include some data types, PG upgrade steps, and parallel-scan integration.
AI-written summary · about
chapters (7)
- 0:00Introduction and background
- 6:30Columnar store basics and features
- 11:41Vertical Clustered Index proposal
- 23:35Performance measurements and results
- 26:53Current implementation details
- 31:30Challenges and future improvements
- 36:01Q&A and conclusion
related talks
46mColumnar Store for PostgreSQL
20mVertica & Column Stores
14mMIT Impact
21mAnalytics in Postgres –A decade in the making
01h 00mColumnar DBMS: The Magic of Data Compression and Super-Fast Queries
01h 03mVertica – High Performance Over Varying Terrain
15mKVIMR: Key-Value Store Aware Data Management Middleware for Interlaced Magnetic...
50mSolving PostgreSQL Wicked Problems