Columnstore Index Compression in a Nutshell
summary
Columnstore storage breaks data into ~2^20-row rowgroups and compresses each column segment separately, using dictionary encoding, value encoding and reordering called vertipack optimization before applying xVelocity (run‑length, bit‑array) compression; typical savings are 10–30:1 and can exceed 100:1. Dictionaries are exposed via sys.column_store_dictionaries and segments via sys.column_store_segments, but dictionaries cap at 16 MB so long strings split rowgroups. Vertipack runs on rebuilds and is skipped for some inserts and memory‑optimized loads.
AI-written summary · about
chapters (10)
- 0:00Introduction to speaker
- 1:00Columnstore index basics
- 2:43Fundamentals of compression
- 3:56Dictionary encoding process
- 5:57Value encoding
- 7:48Vertipaq optimization
- 10:01Physical compression methods
- 12:30Optimizing for compression
- 14:55DMV monitoring demonstration
- 18:33Conclusion and Q&A
related talks
20mColumnstore Index Compression in a Nutshell- 20mColumnstore Index Compression in a Nutshell
01h 00mColumnar DBMS: The Magic of Data Compression and Super-Fast Queries
59mIndex Internals: RowStore, ColumnStore, and NoStore!
16mPolarStore: High-Performance Data Compression for Large-Scale Cloud-Native Databases
46mColumnar Store for PostgreSQL
34mIn core compression-How to shrink your database
26mColumnar Storage: Redefining Data Management for the Modern Era