On compression of everything in Postgres
summary
Postgres now ships multiple codecs—PGLZ, zlib, LZ4, zstd—and shows concrete wins: LZ4 decompression ~5 GB/core vs simple copy ~14 GB/core, making TOAST reads 5–10× faster vs PGLZ on average; compressing full-page images (FPI) with LZ4/zstd yields about ≈10% speedup in pgbench. WAL compression promises big I/O and network savings but hits hard limits: tiny ~<100‑byte records, repeated re-compression by senders/receivers, and much more complex crash recovery. Temporary-file and random‑access compressed‑file prototypes cut written bytes (one patch shows ~4×), but fragmentation and durability make them hard to land.
AI-written summary · about
chapters (13)
- 0:00Introducing Andrey
- 2:00Andrey’s talk begins
- 2:58Basics of compression
- 5:42Pareto frontier
- 7:06pglz
- 8:00zlib
- 8:40Lz4
- 9:34Zstd
- 10:26Compression dictionaries
- 15:30WAL compression idea
- 20:51Protocol compression futures
- 23:15Temporary files compression (future, maybe)
- 27:57Q&A
related talks
34mIn core compression-How to shrink your database
32mBeyond Zstandard: The Future of Data Compression
16mPolarStore: High-Performance Data Compression for Large-Scale Cloud-Native Databases
16mAll the Postgres Things
56mPerformance Improvements in PostgreSQL9 2
25mVindicating ZFS with PostgreSQL: Unleashing the Power of Scalability
51mPostgreSQL on ZFS
35mVindicating ZFS with PostgreSQL: Unleashing the Power of Scalability