Five Adjustments You Need To Do To Your CI/CD Pipeline Before Integrating LLMs In your Application
summary
Five concrete CI/CD changes: add non-deterministic outputs checks and versioned model gating so releases are reproducible; validate structured JSON contracts instead of free text; integrate LLM orchestration tools like LangChain or LlamaIndex and a model zoo router to pick smaller, cheaper models; add continuous bias and security scans (the talk cites Hugging Face growth from ~700k to ~1.1M models and Sonatype showing ~250k malicious package spikes) and produce human-in-the-loop retraining and audit trails.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker intro
- 2:11LLMs changing software interaction
- 6:29LLM integration pitfalls and risks
- 9:49From spaghetti code to modular AI
- 13:38AI in the software lifecycle
- 20:31Human oversight in AI systems
- 23:25Balancing model size and efficiency
- 26:19LLMs in software development
- 29:15Fixing CI/CD pipelines for AI
- 33:54Ethical and security challenges
- 34:58Q&A
related talks
27mThe Lean LLM development playbook
19mLLM codegen fails and how to stop 'em
43mAI Development Lifecycle: Learnings of What Changed with LLMs
17mContinuous deployment for ML: The new software development lifecycle
24mFrom Model to Production: Automating the ML Lifecycle with CI/CD
14mScaling LLM APIs: Integrating Load Testing Into Your CI/CD Pipeline.
29mCustomizing and Evaluating LLMs, an Ops Perspective
38mMaking LLMs reliable - A practical framework