From First Experiments to AI Agents: Adoption Experience and Future Challenges
summary
Describes an on‑prem pipeline that cleans observability data, fine‑tunes an open model on Replicate and serves it with GPTOSS 20B using LoRA adapters and PEFT/Transformers tooling for about $50 per run. Reports the fine‑tuned model produces domain‑toned post‑incident artifacts but lacks company context, so adding RAG with a vector DB (Qdrant/Chroma) improves relevance. Presents a prototype 24/7 agent that pulls logs, opens prioritized tickets and messages on‑call — experiments show faster reports but no clear business‑metric improvement per McKinsey.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:28SRE incident lifecycle
- 3:39Event chain analysis problems
- 5:25First AI automation attempts
- 7:12Model alternatives and fine-tuning
- 9:40Training on historical data
- 11:48Adding context via RAG
- 15:00Fine-tuning and RAG results
- 18:40Pipeline to AI agent
- 20:25AI agent features and future
- 21:49Solution efficiency and impact
- 23:34Q&A
related talks
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability
01h 42mBuilding Responsible AI Agents in Decentralized Open-Source Environments
47mHow to Build Enterprise-Grade AI Agents Using Robust Evaluation
30mThe Great Doubt: What Building an AI Agent Taught Us About Trust
41mGenerative AI: bliss or miss?
45mAI agents in practice - beyond the prototype
26mBeyond Hype: Leading the Charge in Responsible AgentOps for Business Transformation
40mAI Agents in Practice