Autoscaling Your Nomad Jobs Using SRE Insights
summary
Shows a working Nomad autoscaler setup that uses Prometheus metrics and Grafana dashboards to drive decisions, with a demo site that goes from 3 to 1 instance when idle (min=1) and scales to 6 instances to hit an average target of 5 connections per instance (cooldown 20s, max=20). Adds Loki for logs and annotations, and extends to vertical/node scaling by growing AWS ASGs when CPU >70% for 2 minutes (drain deadline 5m). Code and demo repo are runnable.
AI-written summary · about
chapters (13)
- 0:00Introduction and speaker background
- 1:08Evolution of scaling applications
- 3:15Introduction to Nomad job scheduler
- 5:04Deploying a website on Nomad
- 6:28Nomad autoscaler overview
- 9:19Deploying the Nomad autoscaler
- 10:44Prometheus metrics and scaling strategy
- 13:10Monitoring with Grafana dashboards
- 14:18Autoscaling in action demonstration
- 16:46Logging with Loki and Nomad
- 19:57Correlating logs and metrics in Grafana
- 21:42Scaling Nomad worker nodes
- 24:19Conclusion and resources
related talks
27mNomad Best Practices for Reliable Deploys
21mMonitoring Nomad with Prometheus and Icinga
25mInspecting Nomad Events Using Grafana Loki
46mProgressive Deployments with Consul Service Mesh
37mAutoscaling Multi-Cluster Observability with Thanos and Linkerd
40mNomad: Scheduling Applications at Scale
13mAutomate an Observable Runtime with HashiCorp and Ansible
01h 03mScaling Web Operations with Distributed Cluster Schedulers