AIOps: Automating and Scaling ML and LLM Workloads in Production
summary
A complete pipeline builds and runs a private RAG Q&A: Next.js UI calls a FastAPI backend that queries a self-hosted Llama 3.1 8B model with embeddings stored in Chroma DB. Uploading six PDFs triggers six GitLab CI/CD runs that pull docs, chunk and embed them, and hot‑reload the model; Terraform provisions GKE clusters for the frontend, ngrok exposes the LLM, and Prometheus/Grafana monitor metrics. self-hosted LLM RAG pipeline GitLab CI/CD + Terraform + GKE + FastAPI
AI-written summary · about
chapters (9)
- 0:00AIOps introduction
- 3:50Demo: Untrained LLM with docs
- 8:25RAG and LLM architecture
- 13:43Upload docs to trigger pipeline
- 16:14Integrate LLM with frontend/K8s
- 20:55Deploy UI on GKE
- 23:30Monitor LLM and infra
- 26:20DevOps for scalability
- 29:33Resources and Q&A
related talks
43mAI Pipelines With OPEA: Best Practices for Cloud Native ML Operati...
29mBuilding the Bridge Between DevOps and MLOps in a GenAI world
21mMastering the DevOps Machine Learning Pipeline
24mFrom Model to Production: Automating the ML Lifecycle with CI/CD
51mSpeeding up deployment of machine learning models
23mAccelerating Model Deployment with MLOps
53mExperience with an end-to-end MLOps pipeline: Practical insights and 3D architectural perspectives
33mDeploying and Scaling Spark ML and Tensorflow AI Models