Open Source Sovereign AI: Building Enterprise GenAI Platforms for Control & Comp...
summary
An on‑prem GenAI platform built from open source components delivers production‑grade inference, orchestration and observability while keeping data inside the firewall. It uses KServe, vLLM, Triton for serving, MLflow and QFlow for training, Iceberg + a managed lakehouse and vector stores for data, Langraph‑based runtimes and a click‑driven agent builder for democratized apps. Early deployment starts on four A100s → added L40s and H100s. The system is live for internal teams, scaling GPU utilization and meeting compliance and audit requirements.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:05Enterprise AI dilemma
- 4:18Sovereign AI explained
- 6:04Building open-source GenAI
- 8:33GenAI interactions behind scenes
- 10:41Three pipelines of AI
- 11:46Open-source tools used
- 16:28Stack view and Hitachi systems
- 19:00Results and AI democratization
- 20:03Lessons learned
- 21:50Survival guide for AI innovation
- 23:14Q&A
related talks
11mOpen source GenAI vs. enterprise GenAI: Navigating the future of AI innovation
38mIntroducing Open Platform for Enterprise AI
30mSecureAI: A Scalable, Secure, and Compliant AI Solution
09mTransform Your Kubernetes Cluster Into a GenAI Platform: Get Ready-t...
58mAI in Reality Fireside Chat: Enterprise AI & Open‑Source Innovation
24mScalable GenAI in Production
29mNavigating Innovation with Open Hybrid Cloud and Openshift AI
31mOpen-Source AI: Community is the Way.