PolarStore: High-Performance Data Compression for Large-Scale Cloud-Native Databases
summary
PolarStore implements dual-layer compression: a software layer compresses 16 KB pages with per-page adaptive selection between LZ4 and ZSTD (default LZ4 at ~20% CPU, try both when page density >30% using a saved-bytes-per-microsecond threshold), then a CSD hardware layer compresses 4 KB blocks with byte-level indexing via a variable-length FTL. Redo logs bypass compression; a per-page log background merge cuts worst-case page reads from 5+ IOs to 2. Deployed across ~20,000 SSDs, Poly SSD2 + dual-layer reaches 3.55× compression and ~60% storage cost reduction.
AI-written summary · about
related talks
21mPolarDB: Alibaba Cloud-Native Relational Database Service
42mPOLARDB architecture
19mPolyStore: Exploiting Combined Capabilities of Heterogeneous Storage
34mIn core compression-How to shrink your database
32mOn compression of everything in Postgres
14mRe-think Data Management Software Design Upon the Arrival of Storage Hardware with
28mHow Can Databases Capitalize on Computational Storage?
46mColumnar Store for PostgreSQL
