Streaming I/O and vectored I/O
summary
Postgres adds a ReadStream abstraction that groups ReadBuffer calls into larger vectored I/O so scans emit 128k preadv()s instead of many 8k pread()s, using io_combine_limit (default 128k) and effective_io_concurrency to issue concurrent reads; ANALYZE, sequential scans and pg_prewarm already use it in v17. Direct I/O is exposed behind debug_io_direct (PG16) and reveals filesystem quirks (btrfs checksum failures). WriteStreams and background or io_uring-driven asynchronous I/O are WIP to offload and overlap I/O for VACUUM, COPY and CHECKPOINT.
AI-written summary · about
chapters (8)
- 0:00Introduction and overview
- 1:16Historical I/O interfaces
- 9:58Direct I/O in databases
- 14:56Asynchronous I/O concepts
- 20:01Read streams in PostgreSQL
- 33:30Future streaming I/O ideas
- 40:27PostgreSQL 17 and beyond
- 43:34Q&A session
related talks
49mStreaming I/O and vectored I/O
48mThe path to using AIO in postgres
49mAIO in PG 18 and beyond
01h 06mIO in PostgreSQL: Past, Present, Future
26mAdditional IO Observability in Postgres with pg_stat_io
41mLinux IO internals for PostgreSQL administrators in 2023
30mGetting the most out of pg_stat_io
30mDeveloping IO Prefetching for Postgres