Comparing Performance of NVMe Hard Drives in KVM, Baremetal, and Docker
summary
Compares NVMe throughput and IOPS on one Supermicro box with 10 identical Intel NVMe drives attached to a single controller (all in NUMA node 0), measuring baremetal vs Docker vs KVM with fio, SPDK, and automated scripts; observes SPDK yields substantially higher IOPS/bandwidth than libaio, uses PCIe passthrough for VM/container device ownership, and stops long runs early when IOPS slope falls below 3% over 1800s to detect steady state.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:20Define the Problem
- 2:47Warning
- 4:33Three Different Methods
- 8:59PCIe V4
- 10:34NUMA
- 12:20Standard Protocol
- 16:14Testing Tools
- 18:56The Host
- 20:54The List
- 21:22The Plan
- 27:13Torture Test Script
- 28:38Playbooks
- 33:47Testing Example
- 35:00Creating Docker Containers
- 36:16Results
- 41:45Useful Links
related talks
40mAchieving the Ultimate Performance with KVM
26mAnalysis of AMD HW-assisted vIOMMU Implementation and Performance
25mXen and XenServer Storage Performance
21mNVMe Emulation Performance Optimization
40mBenchmarking: Repeatable & Comparable
34mEvaluating the Performance and Power Efficiency of Ceph Storage Configurations
30mSSD and HDD Performance Testing
26mAre You Sure About Your Mesh Performance?