Road to a self-healing infrastructure with HolmesGPT and a private LLM
summary
Builds a private on‑prem, containerized LLM served via OpenWebUI and AMA, wired into Terraform and Kubernetes for code generation and automated repairs; demo shows Continue.dev powering code suggestions (it generates a usable CCE Terraform module) and the private model returning the exact Open Telecom Cloud ingress annotation to restore a public IP. Automation uses HolmesGPT and GPTScript agents plus RAG ingestion (Airbyte → vector DB). Pipeline can auto-open branches/PRs to apply fixes; live demo intermittently fails from network and agent hallucinations, so results work but are not rock‑solid.
AI-written summary · about
chapters (9)
- 0:00Introduction and agenda
- 2:02Private LLM and RAG systems
- 5:40Demo: Private LLM integration
- 7:35Step 1: IDE-based code generation
- 12:14Step 2: Smart CLI agents
- 18:12Step 3: Self-healing infrastructure
- 23:48Complex infrastructure demos
- 30:49Lessons learned and key takeaways
- 33:17Q&A session
related talks
01h 42mBuilding Responsible AI Agents in Decentralized Open-Source Environments
50mFrom Prompt Hacking to Architectural Determinism: Engineering Reliable GenAI Systems
01h 49mAI engineering with open access LLMs that lie, curse, and steal
44mSecuring LLM-Powered Apps: Overcoming Security and Privacy Challenges
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools
30mLLMs on Autopilot: Running AI Agents on Kubernetes with Open Source Tools
01h 15mWorkshop: Working with Gemma and Open LLMs on Google Kubernetes...
23mBuilding Self-Service Development Platforms