How to Survive to High Demanding Services
Talkat ContainerCon @ OSS EU 2023 · on Wednesday, 2023-09-20· at 12:20· Room 0A (Floor 0), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Handles ~400,000 requests over 12 hours from ~30,000 IPs using GPU-backed AI models on Kubernetes, logging via a fluentd sidecar with custom Node script that ships to Datadog dashboards and fires Teams webhooks for errors and timeouts (example: 60s observed vs 15s threshold). Istio service mesh provides rate‑limit rules with two Redis-like stores (per‑second and per‑minute), returning 429s under abuse, plus shadow (mirror) and canary traffic via VirtualService/DestinationRule and Jenkins pipelines to validate releases with real traffic; solution works in production.
AI-written summary · about
chapters (7)
- 0:00Surviving high demand services
- 2:06Monitoring service metrics
- 9:26Real-time error detection
- 13:18Denial of service protection
- 20:18Shadow deployments
- 22:39Canary releases
- 32:22Summary
related talks
34mHow to Survive to High Demanding Services
14mMonitoring Network Traffic in Kubernetes
41mKubernetes the Very Hard Way
24mIstio in Production: Expected Value, Results, and Effort
49mComprehensive Container-Based Service Monitoring with Kubernetes and Istio
37mThe Service Mesh Wars: A New Hope for Kubernetes
45mIstio to the Rescue
44mKubernetes at Datadog Scale