Empowering ML Workloads With Kubeflow: JAX Distributed Training and LL...
summary
Extended Kubeflow implements a tune API for LLM HPO that wraps the trainer SDK, storage initializer and PyTorchJob so experiments pull models from Hugging Face/S3, spawn distributed trials, and return best hyperparameters (demo runs with 8 Yelp examples, max_trials=1, parallel=1 and reports best learning_rate and r). Kubeflow Training Operator adds JaxJob CRD v1.9.0, uses kind, launches two worker replicas with glue for collectives, calls jax.distributed.initialize and completes successfully showing near-linear scaling as workers increase.
AI-written summary · about
chapters (5)
- 0:00ML workloads with Kubeflow
- 1:25LLM hyperparameter optimization
- 10:29JAX distributed training
- 17:38Key takeaways
- 18:45Q&A
related talks
16mStreamline LLM Fine-tuning on Kubernetes With Kubeflow LLM Trainer
36mStreamlining ML Workflows With the Unified Kubeflow SDK
21mStreamline LLM Fine-Tuning on Kubernetes with Kubeflow LLM Trainer
14mKubeflow: Where ML Meets Kubernetes
31mBuilding and Managing a Centralized ML Platform with Kubeflow at...
31mKubeflow 2026 State of the Union: Orchestrating the Full ML Lifecycle
51mMachine Learning using Kubeflow and Kubernetes
46mDeploying scalable Machine Learning models with Kubernetes