Emulate Block Device Incidents With IO Scheduler
summary
Describes a Linux IO scheduler called IOEM kernel module that emulates block-device latency and IOPS per device and per process, letting teams reproduce a real fault where fault‑tolerant logic failed; implements timing via an RB‑tree and a global traffic limiter, dispatching requests only when scheduled. Example test limits 1005 requests in 100 microseconds and yields about 14–15k IOPS, matching expectations. Works as a loadable kernel module (CentOS 7 up to 5.15 kernels), integrates with Chaos Mesh for Kubernetes, and is removable or switchable at runtime unlike device‑mapper delay.
AI-written summary · about
chapters (9)
- 0:00Start
- 0:44Why does it matter?
- 3:21Linux 10 (from application perspective)
- 5:14Inject latency through ptrace
- 7:36Error Injection File System
- 10:21Linux 10 (from driver perspective)
- 11:13Multi-Queue Scheduler
- 15:39Inside Container / Cloud
- 17:26Current Status
related talks
40mLinux Storage System Bottleneck for eMMC/UFS
45mData Placement in Linux: Evolving Block and File I/O
36mIntroducing the libblkio High-performance Block I/O API
30mDeveloping IO Prefetching for Postgres
37mLarge Block I/O for Linux
01h 06mIO in PostgreSQL: Past, Present, Future
34mEnhancing Data Integrity in Linux
51mWhat went wrong with AIO