Columnar Table Performance Enhancements Of Greenplum Database
summary
Alibaba extends Greenplum (HybridDB) with block-level min/max metadata and disk sort keys, implemented inside columnar files and generated on-the-fly metadata generation with about a 10% load cost. The metadata lets the engine skip many 1 MB blocks so metascan often matches or beats index scans and in some GROUP-BY/ORDER-BY cases shows >2x speed versus sequence scans. Sort keys eliminate re-sorts for certain GROUP-BY/ORDER-BY workloads and yield near-constant response time.
AI-written summary · about
chapters (12)
- 0:00Introduction and background
- 1:05Alibaba Cloud overview
- 2:25Greenplum architecture
- 5:48Harbor ADB and challenges
- 7:43Columnar table limitations
- 10:33Structure and storage
- 13:45Transactions and performance
- 18:58Metadata scan feature
- 24:41Sorting keys and optimization
- 28:19Multidimensional search
- 31:36Performance benchmarks
- 36:02Conclusion and Q&A
related talks
26mColumnar Storage: Redefining Data Management for the Modern Era
30mGreenplum Database — massive parallel processing with a shared-nothing architecture based on Postgres
35mFrom Stars to Storage Engines: Migrating Big Science Workloads Beyond Greenplum
37mBenefits of columnar data layout
01h 00mColumnar DBMS: The Magic of Data Compression and Super-Fast Queries
42mColumnar Storage: Redefining Data Management for the Modern Era
45mClickHouse: what is behind the fastest columnar database
41mCompute/Storage Separation in Greenplum