Rollout on Reception: Progressive Delivery for GenAI Apps Using Real...
summary
Live demo shows users scanning a QR, voting thumbs up/down every 30 seconds and Knative Services + Flagger adjusting canary weights (start 20% → 40% → 50%) in real time; Flagger polls Prometheus metrics converted from OpenTelemetry traces by a collector using a transform processor and a spanmetrics processor. Platform stacks include Spring AI, Aronia, PostgreSQL, Flux, Crossplane and Datadog LLM observability. A cheap "mistro" model beat a pricier Anthropic model, so user votes directly decide rollout or rollback.
AI-written summary · about
chapters (10)
- 0:00GenAI rollout challenges
- 1:04User-feedback rollout demo
- 4:30Demo results and prompt variations
- 7:17OpenTelemetry GenAI conventions
- 11:30Trace data to Prometheus metrics
- 13:34Flagger and KNative architecture
- 16:22Instrumenting GenAI with Spring AI
- 17:25Model comparison demo
- 20:40Key takeaways and cost savings
- 21:44Q&A
related talks
24mScaling on Satisfaction: Automated Rollouts Driven by User Feedback - Thomas Vitale & Whitney Lee
26mBeyond Argo Rollouts: Boosting Developer Experience With Intelligen...
24mFix Production Rollouts on the Fly With Agentic AIOps
42mEffortlessly Build, Run, Secure, and Manage Traffic for a Generative AI Application From...
35mProgressive Delivery with Argo Rollouts
11mKeynote: Smarter Progressive Delivery at Intuit
33mOperationalizing GenAI Applications with DevOps Practices
40mProgressive Delivery with Kubernetes and Argo Rollouts