Kubernetes AI + HPC Day North America 2023
2023-11-06
Chicago, United States
| title | author | length |
|---|---|---|
| Welcome + Opening Remarks, Program Committee Members | Ricardo Rocha, Yuzhui Liu | 06m |
Event opens with logistics: code of conduct, AI-generated captions via QR, and snacks/lunch pickup points; CFP for KubeCon Europe in Paris opens and closes Dec… | ||
| Building a Multi-Cluster Privately Hosted LLM Serving Platform on Ku... | Julian Bright, Noah Yoshida | 26m |
Built a multi-cluster, privately hosted LLM serving platform on Kubernetes that uses Istio service mesh for a central control plane and per-customer data… | ||
| How We Power the Largest AI Deployments on the Planet: Running Vir... | Brandon Jacobs, Lukas Gentele | 25m |
CoreWeave runs thousands of Kubernetes control planes across bare metal—3,000+ customer clusters on ~5,400 nodes and tens of thousands of GPUs—by virtualizing… | ||
| Batch Systems in Production with Kueue: Multi-Tenancy and Fungibility | Yuki Iwai, Aldo Culquicondor | 33m |
Kueue is a Kubernetes-native queueing layer that builds multi-tenant batch clusters by enforcing quotas with borrowing and preemption and mapping workloads to… | ||
| Cluster Operations as a Service, Introducing LLM Backed Controllers. | Rajas Kakodkar, Amine Hilaly | 31m |
A prototype controller called LMNities controller uses LLM inference (GPT‑3.5 via OpenAI API) and fine‑tuned models (TinyLlama, Llama 2) to turn natural… | ||
| Scheduling Jupyter Notebooks Using Airflow on Kubernetes | Akshay Chitneni, Rus Pandey | 28m |
Jupyter notebooks are run as on‑demand remote kernels on Kubernetes using an extended kernel spec (Docker image, CPU/memory, cluster/namespace, data access… | ||
| Lightning Talk: Harnessing Generative AI Without Making a Faux Pas | Shwetha Subramanian | 06m |
Demonstrates a locally hosted FauxPilot setup plus an NVIDIA inference server and FasterTransformer backend to generate complex Kubernetes YAML blueprints… | ||
| Lightning Talk: Environmentally Sustainable AI via Power-Aware Batch Scheduling | Atanas Atanasov | 07m |
Demonstrates a prototype that improves performance per watt for LLM training by controlling node component power and batch scheduling: integrates with Kueue/Q… | ||
| Lightning Talk: SKAO: Next-Generation Radio Astronomy-Driven Big Data | Ugur Yilmaz | 06m |
SKAO plans an observatory with 2 telescopes producing raw radio streams at terabytes per second and ~9 TB/s heading to science pipelines, requiring around 40… | ||
| Improving GPU Utilization and Accelerating Model Training with Scheduling Framework and NRI | He Cao | 24m |
Improving GPU Utilization and Accelerating Model Training with Scheduling Framework and NRI - He Cao, ByteDance
K8s is no longer just running service-oriented… | ||
| Training Large Language Models on Kubernetes | Ronen Dar | 28m |
Shows Kubernetes running multi-node LLM pretraining (Megatron/NVIDIA Nemo) on A100 DGX gear and compares bare metal vs K8s throughput, reporting <2% overhead… | ||
| Keep HPC Running - an SRE's Guide to Supporting GPUs on Kubernetes | Christopher Dutra | 22m |
Shows a working approach that queries DCGM-Exporter metrics (via NVIDIA DCGM and NVML bindings) from Prometheus and implements a custom node-problem-detector… | ||
| Poisoned Pickles Make You Ill | Adrian Gonzalez-Martin | 27m |
Pickle serialization is widespread in scikit-learn, Keras and PyTorch and it lets arbitrary Python objects — including functions — execute on load, so poisoned… | ||
| Constructing Kubeflow Super Nodes with Karpenter | Roland Barcia, Apoorva Kulkarni | 29m |
Karpenter automatically provisions and consolidates nodes for Kubeflow notebooks and pipelines, launching instance types (spot or on‑demand, Graviton/AMD) to… | ||
| Training Foundation Model Workloads on Kubernetes at Scale W... | Abhishek Malvankar, Olivier Tardieu | 26m |
Vela runs on Kubernetes with about 1,000 A100 GPUs and aims for near‑zero idle capacity; MCAD queues AppWrapper CRDs and dispatches heterogeneous frameworks… | ||
| Lightning Talk: Multi-Party Learning Using Homomorphic Encryption with Kubeflow... | Aakanksha Duggal | 06m |
Describes an implemented workflow that uses Kubeflow Pipelines orchestration to preprocess private datasets locally, encrypt them with homomorphic encryption… | ||
| Lightning Talk: Multiplayer Machine Learning with Metaflow, OpenAI Whisper and... | Shrinand Javadekar | 10m |
Demonstrates using Metaflow with OpenAI Whisper on Kubernetes to transcribe three audio URLs in parallel, running tiny and small Whisper models per file and… | ||
| Lightning Talk: Optimize the Ray Schedule and Autoscaling on Kubernetes Cloud: A Hetero... | Liang Yan | 09m |
Describes a concrete optimizer for Ray on Kubernetes that replaces uniform thresholds with a task-type-aware scheduler and autoscaler that chooses VM flavors… | ||
| Closing Remarks, Program Committee Members | Ricardo Rocha, Yuzhui Liu | 01m |
Closing remarks note that this is Day Zero and ccon starts tomorrow with two concurrent tracks: emerging and advanced concepts (HPC and scientific/research… | ||
| nothing matches | ||