Extreme infrastructure automation with Wavefront

on YouTube ↗46m

" summary

Wavefront runs telemetry at scale (hundreds of billions of points/day, 25‑month retention) on Spotinst’s platform and AWS, using Ocean and ElasticGroup to match pods to instances, preemptively replace soon‑to‑be‑reclaimed EC2 Spot Instances, and fall back to reserved or on‑demand automatically. The deployed system covers ~5,000 containers across 12 clusters for ~500 users with 600 dashboards and 700 alerts, claims ~70% cost reduction in a sample cluster, and restores capacity within ~20 minutes while keeping clusters schedulable.

AI-written summary · about

" chapters (62)

1. 0:00Intro
2. 1:10What we do
3. 2:05Cloud Resource Hours
4. 2:28Spot Instances
5. 5:59OnDemand Instances
6. 7:34Ocean
7. 9:28Why containers
8. 11:08Container driven autoscaling
9. 12:08Call show back
10. 13:24Containers
11. 15:05AWS
12. 15:44Putting it all together
13. 16:35Platform talk
14. 17:07About Wavefront
15. 17:54Journey for Wavefront
16. 18:24Wavefront Effect
17. 19:32Making Sense of the Data
18. 19:53Beach Ops
19. 20:50Red metrics
20. 21:17Developer logs
21. 21:38Keep your customers happy
22. 21:51Single scope view
23. 22:09Time series view
24. 22:29Visualizations
25. 23:00Correlation
26. 23:44Architecture review
27. 23:54Why Wavefront
28. 25:44Architecture diagram
29. 26:30Code view
30. 26:51Progress over perfection
31. 27:33Pando tree
32. 27:47Root tree
33. 28:32Run per cluster
34. 28:42Trunk
35. 29:46Run once per cluster
36. 30:08Optional modules
37. 30:42Clusters
38. 31:15Build a Tree
39. 32:12Build Trees Across Regions
40. 32:30Build Trees in Separate Accounts
41. 32:58Support Multiple Customers
42. 33:16SelfService Portal
43. 34:03Subdirectories
44. 35:40Creating a new cluster
45. 36:15Building the cluster
46. 37:07Spot Integration
47. 38:29Hidden Value
48. 39:18Focus on Innovation
49. 39:35Cost Savings
50. 40:07Cost and Cost
51. 40:19Breaking out by namespace
52. 40:34Right sizing
53. 41:08Node views
54. 41:24Logs
55. 41:37Wavefront UI
56. 42:16Wavefront Dashboard
57. 43:11Robust Query Language
58. 43:29Interactive Query Builder
59. 43:59Helpers
60. 44:22Cluster savings
61. 44:41Wavefront Unified View
62. 45:30Contact

" related talks

• 47mReliable, secure, and efficient cloud operationsKevin McGrath · AWS re:Invent 2022
• 28mHarness cloud elasticity by bursting to Amazon EC2 SpotViral B Shah, Graeme Demmocks · AWS Summit ANZ 2023
• 26mRunning FDB at scale at WavefrontMatthew Zeier, Mike McMahon · FoundationDB Summit 2018
• 34mKubernetes: Reducing Infrastructure cost and complexityKevin McGrath · ContainerDays 2022
• 42mAWS Spot instances: huge savings (or huge risks?)Taras Postument · code.talks 2019
• 43mSpot InstancesDerek Felska · DevOps World | Jenkins World 2019 San Francisco
• 39mHow automating cloud operations accelerated time to market at NUKevin McGrath, Vitor Puente · AWS re:Invent 2023
• 29m#BuildingTheFuture: Keeping our Dutch Feet Dry: Setup and Operation of a...Onno Brouwer · Cloud Foundry Summit Europe 2019