Streaming I/O and vectored I/O
summary
PostgreSQL adds a streaming abstraction and vectored I/O so multiple 8KB blocks are combined into larger reads/writes (configurable IO combine limit, default demos at 128KB) and can be prefetched or issued asynchronously. Implementation uses read streams, fadvise-based “poor man” async now and three async modes (sync worker, I/O worker processes, and IO_uring), changes landed in PG17 and ongoing for PG18. Benchmarks on a 72GB table show roughly 2x faster sequential scans and far fewer CPU cycles with direct I/O plus async workers.
AI-written summary · about
chapters (11)
- 0:00Introduction to I/O basics
- 3:15Direct I/O in databases
- 5:39Implementing vectored I/O
- 7:21Need for asynchronous I/O
- 9:25Read streams architecture
- 25:24Experimental I/O research
- 30:57Asynchronous I/O subsystem
- 34:42Testing AIO patches
- 38:31Performance benchmarks
- 39:31Conclusion and future work
- 40:44Q&A session
related talks
48mStreaming I/O and vectored I/O
48mThe path to using AIO in postgres
49mAIO in PG 18 and beyond
01h 06mIO in PostgreSQL: Past, Present, Future
41mLinux IO internals for PostgreSQL administrators in 2023
30mDeveloping IO Prefetching for Postgres
30mGetting the most out of pg_stat_io
26mAdditional IO Observability in Postgres with pg_stat_io