From Stars to Storage Engines: Migrating Big Science Workloads Beyond Greenplum
summary
Archives hold about 1 PB of files and ~400 TB of databases and Gaia DR3 supplies ~2 billion rows used to test scale‑out options. Greenplum worked for MPP and columnar compression but Broadcom closing the source forces alternatives: PoCs run against Citus (Postgres 17, fast parallel ingest) and EDB WarehousePG (Greenplum-compatible binary swap). Key blockers are extension support (Q3C spherical index), distributed join performance, operational tooling and vendor support; evaluations continue without a final migration chosen.
AI-written summary · about
chapters (8)
- 0:00Introduction and background
- 3:10ESA’s data mission and challenges
- 7:02Scaling limits of Greenplum
- 9:54Evaluating scale-out solutions
- 15:37Vendor lock-in and alternatives
- 22:43Migration costs and support needs
- 25:53Final evaluation and conclusions
- 28:22Q&A session
related talks
30mGreenplum Database — massive parallel processing with a shared-nothing architecture based on Postgres
59mUsing Postgres XL
46mBig databases, many servers, data center or cloud - it's all here!
40mLife After Greenplum: Choosing an Open Source Analytics Solution
36mMigrating Our Data Warehouse to the Cloud
36mMigrating a Data Mart from Teradata to Greenplum
53mFrom Greenplum to Apache Cloudberry
39mColumnar Table Performance Enhancements Of Greenplum Database