Ceph Crimson Verification on AArch64
summary
Built ARM Crimson images and deployed a three-node cluster with cephadm + fio to benchmark c-store; discovered a memory-leak pattern in buffer lists that can cause OOM and plan buffer-list and memory-pool refactors. Identified a ~35% CPU hang in memory-port atomics using ARM LSE; disabling LSE and enabling 2MB huge pages yields about ~50% improvement for small I/O and ~8% for larger I/O. Found 1MB extent allocation in c-store causing heavy read amplification and has PRs to limit extent size and enable partial-extent reads.
AI-written summary · about
chapters (5)
- 0:00Introduction to Crimson
- 1:25Memory leaks and waste
- 2:50Performance bottlenecks
- 4:39Multi-node cluster testing
- 9:45Q&A session
related talks
24mLatest storage status on Arm
21mKeynote: Ceph Developers AMA: Meet the Minds Behind Open Source Ceph
24mReliability, Availability, and Serviceability (RAS) on ARM64 status
45mIn-depth technical story: Fixing I/O performance for Windows guests in OpenStack Ceph clouds
19mMicroCeph: Simplifying Storage
30mEncrypting Memory at Scale
45mUsing hardware hints for optimal page placement
36mDebugging with eBPF on Arm Platforms