Scaling on Satisfaction: Automated Rollouts Driven by User Feedback - Thomas Vitale & Whitney Lee
summary
A live platform captures thumbs up/down as correlated OpenTelemetry spans and span events, the Collector’s transform and spanmetrics processors convert those into Prometheus metrics, and Flagger + Knative drive canary routing from 20% initial traffic through automated 30s steps over a 4‑minute rollout. Crossplane and CloudNativePG provision services; DataDog shows traces and 134 votes in the demo. Two LLMs (Mistral vs Anthropic Opus) compete and the rollout stops when user feedback indicates the cheaper model suffices, saving cost.
AI-written summary · about
chapters (9)
- 0:00User feedback in AI apps
- 1:03Live story variant voting
- 5:24User votes to rollout decisions
- 7:48Votes to OpenTelemetry metrics
- 13:02Trace data to actionable metrics
- 15:09Dynamic scaling architecture
- 18:43Java app with Spring AI
- 20:06Live demo user rollout
- 22:35Key takeaways
related talks
24mRollout on Reception: Progressive Delivery for GenAI Apps Using Real...
45mPerformance-by-Design: Embedding Intelligent Scaling and Guardrails Into Platform Engineering
38mPerformance Testing of Prometheus Based Metric Platforms
46mProgressive Deployments with Consul Service Mesh
16mProgressive Delivery with Observability
40mDeploying OpenTelemetry Organizationally: From Proof of Concept to In-Production at Scale
33mProgressive Delivery Techniques with Flagger
21mAutoscaling and Progressive Delivery - A Match Made in Heaven