Making Complex R Forecast App...
summary
Built a Forecast-as-a-Service that keeps R for modeling but moves orchestration to Argo Workflows on Kubernetes, using GCP Pub/Sub, Prometheus/Grafana and CI/CD. A Python decoder divides jobs and parallelizes 100s of series, lightweight custom Docker images cut build time from three hours to five minutes, and Argo enforces retries, pod limits and cost-aware autoscaling. Monitored multi-region private clusters now run mixed R and Python pipelines, improve debugging with centralized logs, and reduce runtime and cost compared with the monolith.
AI-written summary · about
chapters (10)
- 0:00Intro
- 0:51Challenges
- 1:50Getting there
- 3:06Gains
- 4:37New Infrastructure
- 5:11R Pros and Cons
- 6:07Solution Needs
- 6:31Decoder
- 7:02Image
- 8:04Architecture
related talks
42mRobust Automated Forecasting in Python and R
10mScaling Time Series Analysis With Argo Workflows: Patterns and Practi...
25mPanel: Argo for ML: Achieving Scalability and User Experience
07mArgo in an AI-Native World: Effective In-Cluster Machine Learning and A...
23mMastering Argo Workflows at Scale: A Practical Guide to Scalability Excellence
25mLeveraging Argo Workflows and Argo Events for Cloud Agnostic Enter...
26mLeveraging Argo Ecosystem for Scalable Cloud-Native Solutions at Databricks
27mEmpowering Autonomy: BYD's Journey Taming Million-Task Scale With Argo...