Building a Global Supercomputer with Virtual Kubelet
Talkat KubeCon + CloudNativeCon North America 2020 Virtual · on Thursday, 2020-11-19· at 14:55· Intrado Virtual Event Platform,·
summary
Nautilus aggregates donated campus and international hardware into a single Kubernetes fabric and runs Admiralty plus Virtual Kubelet to federate many clusters, currently exposing 7,000 CPU cores, 2.5 PB storage and 500+ GPUs as virtual nodes. Jobs are scheduled as mutated proxy pods that create candidate and delegate pods in target clusters; GitLab CI runners burst to remote clusters in the demo. Cross-cluster auth, per-namespace control, and on-demand Layer‑2 paths enable decentralized governance and scalable bursting toward a theoretical 100,000‑core federation.
AI-written summary · about
chapters (12)
- 0:00Start
- 0:15Outline
- 0:38The Early Days of the Pacific Research Platform
- 7:54Immediate Use Cases of Federation
- 10:28Future of Federation
- 17:13Example Use Case
- 18:36Virtual Kubelet Responsibilities & Admiralty Implementation
- 20:17Cluster Connections
- 21:18Cluster Topologies
- 21:57Cross-Cluster Authentication Methods
- 25:27Admiralty's "Look-Ahead" Multi-Cluster Scheduling Algorithm
- 29:27Summary
related talks
30mThousands of Virtual Kubelets: 1-to-1 Mapping a Supercomputer To Kubernetes With Supernetes
31mManaged Kubernetes — Next Gen Academic Infrastructure?
41mBuilding & Operating a Large-scale HPC AI Cluster on Kubernetes
34mA Huge Cluster or Multi-Clusters? Identifying the Bottleneck
19mKeynote: Building Bridges: Cloud Native and High Performance Computing
37mOne Cluster to Rule Them All - Multi-Tenancy for 1k Users
32mPods Everywhere! InterLink: A Virtual Kubelet Abstraction Streamlining HPC Resource Exploitation
37mVirtual Clusters in the Real World: Lessons from k3k’s Journey to Production