Lessons learnt expanding Cambridge University's CSD3 Supercomputer with OpenStack
summary
Cambridge expands CSD3 using Ironic bare-metal provisioning on a ~1000‑node pool and Dell PowerEdge C6420 servers with Mellanox ConnectX‑6 (HDR100/50G breakout). Automation reads rack CSVs, creates Neutron ports via Terraform, builds Ansible inventories, and enrolls nodes into ironic using IPMI and inspection RAM disks. Direct deploy over HTTP plus tuning (force_raw_images=false, disable power sync, raised timeouts) and a networking generic switch driver with xcd batching cuts switch reconfigure time to under a minute and enables reimage 56 servers in 20 minutes, proving OpenStack can run large‑scale HPC though half the cluster still uses xCAT.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 0:23CSD3 overview
- 1:29Increasing complexity
- 2:49Hardware silos
- 3:24Openstack
- 4:36Cambridge HPC Services
- 4:46History of Cambridge HPC Services
- 5:29Arcus Cloud
- 6:16Hardware
- 7:31Finding the machines
- 10:09Building the cluster
- 13:12Tuning for scale
- 20:52Monitoring
- 22:57Whats next
related talks
15mThe CERN Cloud Infrastructure: 5 years in 15 minutes
32mNo Touch Installation Concept
22mMoving a Large Workload from a Public Cloud to an OpenStack Private Cloud...
19mBuilding at Scale with H100: Eos as a DGX SuperPOD Reference Model for Large Data Center Builds
20mKeynote Welcome
26mOpenHPC on Ansible
22mEnabling Cloud Foundry to Run on Bare Metal
33mIntent-based holistic data center management