Exposing PCIe topology to Guest OS for peer-to-peer
summary
Problem: guest kernels lack real PCI topology, ACS and root-complex type so the p2pdma check (map_type_list / “calc”) rejects or behaves unpredictably for VM peer-to-peer. Proposals: replicate host topology with qemu emulated root port and virtual PCI switch; implement a virtio topology device that runs calc in user space; expose a user API to call calc from VM setup; or allow a kernel command-line override that forces p2p allowed. Current vfio behavior bypasses calc so some configurations already work but remain unsafe; no single solution is yet built end-to-end.
AI-written summary · about
chapters (11)
- 0:00Introduction to PCIe P2P
- 1:03Current kernel limitations
- 6:28Problem with guest OS exposure
- 10:01Proposal one replicate topology
- 14:13Security concerns with topology
- 17:26Missing pieces in user space
- 21:00How P2P DMA works in QEMU
- 23:26ATS and security implications
- 24:53Simple kernel command line solution
- 26:18Proposal two virtio device
- 27:19Q&A and discussion
related talks
41mVirtIo-Net PCIe Function Using Linux PCI Endpoint Framework- 41mVirtIo-Net PCIe Function Using Linux PCI Endpoint Framework
36mShared Virtual Addressing for High Performance Arm Infrastructure Platforms
30mVirtio for PCI Endpoint Subsystem in Linux Kernel
25mChallenges in Supporting Virtual CPU Hotplug on SoC Based Systems (like ARM64)
21mVirtio Devices Emulation in SPDK Based on Vfio-User Protocol
28mVirtio on Type-1 hypervisors
24mvDPA Support in Linux Kernel