Lessons from Optimizing Citus Queries at Heap

on YouTube ↗23m

" summary

Heap uses confidence intervals to prove performance changes are real, not noise, and turns index-only scans into real wins by either including the expression in the index or by creating a matching partial index that yields a 2x P90 win on a real query. A metadata bug during a distributed schema migration breaks ingestion; recovery uses simple SQL to replay rows from audit tables into the coordinator mapping so production stays running.

AI-written summary · about

" chapters (6)

1. 0:00Introduction
2. 1:26Performance challenge @ Heap
3. 3:22Define confidence intervals & apply to experiments
4. 8:18How to get index-only scans on column expressions
5. 14:17Use audit tables to identify problems
6. 22:00Recap

" related talks

• 38mAccelerating Query Performance in CrateDBMarija Selakovic · Big Data Conference Europe 2023
• 01h 09mDesigning the Right Schema to Power HeapDan Robinson · PGConf Silicon Valley 2016
• 53mIndexing Internals for Developers & DBAsBradley Ball · SQLBits 2022
• 53mIndexing Internals for Developers & DBAsBradley Ball · SQLBits 2024
• 50mImproving Postgres' EfficiencyAndres Freund · PostgresOpen Silicon Valley 2017
• 31mMove Less, Move Faster: Speeding Up Citus Cluster ScalingMuhammad Usama · POSETTE: An Event for Postgres 2026
• 26mWhere do the performance cliffs come from?Tomáš Vondra · POSETTE: An Event for Postgres 2024
• 23mWhen ms matter: Maximizing query performance in CrateDBMarija Selakovic · Berlin Buzzwords 2023