IO in PostgreSQL: Past, Present, Future
summary
Prototype replaces kernel pagecache I/O with an asynchronous layer using io_uring backend plus a worker-process fallback for portability, changes checkpointer, background writer and reads, and adds a streaming-read callback and scatter/gather coalescing. Measured wins include reaching about 12 GB/s checkpointing on PCIe NVMe and up to 2.5x faster per-core sequential scans; WAL flush concurrency improves but can inflate WAL volume 3–4x in worst cases. Remaining work: portability, padding heuristics, and a better buffer-replacement policy.
AI-written summary · about
chapters (16)
- 0:00Intro
- 1:24Design
- 4:45Buffer Replacement
- 5:55Scan Flooding
- 8:27Checkpointer
- 10:38Why didnt PostgreSQL fail
- 13:32Why are we still doing this
- 19:31Hardware landscape has changed
- 24:08Increased bandwidth
- 26:38Cloud storage latency
- 29:14AIO in PostgreSQL
- 30:38Design constraints
- 33:50Current design
- 39:06Conversions
- 41:33Checkpointing
- 53:07Algorithms
related talks
41mLinux IO internals for PostgreSQL administrators in 2023
48mThe path to using AIO in postgres
49mAIO in PG 18 and beyond
49mStreaming I/O and vectored I/O
30mDeveloping IO Prefetching for Postgres
48mStreaming I/O and vectored I/O
51mWhat went wrong with AIO
26mAdditional IO Observability in Postgres with pg_stat_io