Less Tokio is More Tokio: Strategies for Accelerating a Rust Native Search Database
summary
A Rust-native storage layer on top of Tokio redesigns the I/O loop into a synchronous submit/poll flow and a push/pull task queue, removing an extra submitter and drastically reducing handoffs. The team adds io_uring batching, starts I/O when scheduling rather than inside task closures, and moves to fewer tokio tasks with eagerly polling futures. The result reaches about 3,800 QPS and sustains around 1.8M reads/sec, showing the approach works.
AI-written summary · about
chapters (16)
- 0:00Start
- 0:15Introduction to Lance DB and the speaker
- 2:02Why this talk isn’t about IO_uring
- 3:07What Lance DB does and its use cases
- 8:42Why vector search requires millions of IOPS
- 10:27Balancing horizontal and vertical scaling
- 11:11Challenges of achieving high IOPS on NVMe
- 15:10Understanding synchronization overhead in IO
- 20:22Why simply adding IO_uring didn’t work
- 23:08The original IO scheduling problem
- 26:18Simplifying the scheduler to reduce overhead
- 28:25Integrating IO_uring for batching IO requests
- 30:31Benchmarking setup and hardware goals
- 32:40Results and unexpected performance gains
- 34:10Lessons learned from the IO_uring API
- 36:39Key takeaways for high-ops workloads
related talks
29mI/O in Rust: the whole story
06mZero-copy Transfers with io_uring and Rust
05m“Using async-profiler to Find Tail Latency Issues”
34mWhoops! I Rewrote it in Rust
29mAsychronous programming in Rust
51mThousands of Async Tasks per Second in the Cloud on Rust/Axum/Tokio
27mWhoops! I Rewrote It in Rust
56mThe RUST Language for High‑Load Network Services — Quick Start