BOSH & GPU - a Match Made in Heaven
Talkat Cloud Foundry Day North America 2024 · on Wednesday, 2024-05-15· at 11:20· Education Center, 2nd Floor·
summary
Demonstration of running GPUs under BOSH shows a working proof: modified CPIs enable GCP Tesla T4 and AWS G4 instances, vSphere supports both passthrough and vGPU with vendor/device IDs, and driver install uses simple pre-start scripts after a root disk expansion (feature merged in 97.06). A RAG demo uses an OpenAI-compatible server with embeddings from Cloud Foundry docs and successfully answers queries. Remaining gaps: EFI boot for >16GB GPUs and broader Azure/OpenStack validation; sample BOSH release and configs are on GitHub.
AI-written summary · about
chapters (13)
- 0:00Introduction and background
- 1:27GPU support in BOSH
- 3:14Cloud provider GPU setup
- 4:37GPU driver installation challenges
- 6:44GPU drivers and root disk expansion
- 7:25GPU in Diego containers limitations
- 8:50Large language models explained
- 9:33LLM as a service concept
- 10:59Live GPU demo on BOSH
- 12:43Demo results and applications
- 13:05EFI support for GPUs
- 14:07Future GPU workloads and opportunities
- 14:27Q&A session
related talks
36mOpenPOWER and GPU Porting Platform for FOSS Using OpenStack at OSUOSL
33mUnleash the Multi-Cloud with BOSH
05mDay 1 Keynote
47mGet off my GPU! Better Sharing of AI Accelerators in OpenStack with Blazar Reservations
18mArchitecting Multi-tenant Data-center Networks for GPU Customers
37mVibe Computing: AI-Powered Multi-Cloud Orchestration With Cloud-Barista
01h 13mEngineering the Cloud for Tomorrow: Performance, Speed and Scale for AI Workloads
06mAutoscaling GPU Clusters Anywhere — Hyperscalers, Neoclouds & Ba...