How Yelp.com Runs on Apache Mesos in AWS Spot Fleet for Fun and Profit

on YouTube ↗36m

" summary

Yelp runs Mesos clusters on AWS Spot Fleet, saving about 50% compared to reserved instances by overprovisioning and accepting 120‑second terminations. Engineering delivers this with a Terraform module that expresses diverse AZ+instance combinations, per‑AZ Spot Fleet requests, and a spot-aware autoscaler that rebalances by preferring instances with fewer tasks. Mesos/Marathon handle fast rescheduling; maintenance hooks (instance termination webhook → maintenance mode) exist but aren’t fully relied on due to a Mesos bug.

AI-written summary · about

" chapters (40)

1. 0:00Introduction
2. 1:16Agenda
3. 1:36Spot Fleet Definitions
4. 2:24Spot Instances
5. 4:01Spot Price
6. 4:44What is Spot Fleet
7. 5:42Why is Spot Fleet important
8. 6:41Spot Fleet Web Interface
9. 7:22Spot Fleet CLI
10. 8:09The JSON Problem
11. 9:02Terraform
12. 9:49Why use Terraform
13. 10:42Spot Fleet Instance Types
14. 11:52Why We Use Terraform
15. 12:48Terraform Demo
16. 13:17Best Practices
17. 13:35Diversification
18. 14:26Diversification vs Lowest Price
19. 15:33Diversification vs Rebalance
20. 15:53AZ Instance Types
21. 16:12Diversification Weighting
22. 16:43Diversity in Action
23. 17:36How to Achieve Diversity
24. 17:57Easy Balancing
25. 20:08Bidding
26. 22:41Dealing with Terminations
27. 24:28Outbidding Yelp
28. 25:41Maintenance Primitives
29. 26:06Marathon
30. 26:21Outbid
31. 27:49Maintenance
32. 28:18Recap
33. 29:31Is it worth it
34. 29:42I ran the numbers
35. 30:00Spot Fleet is saving us tons
36. 30:38Spot Fleet is saving us 50
37. 31:31Shout out to the early adopters
38. 31:56What you need to know
39. 32:28GitHub
40. 33:14Costbenefit analysis

" related talks

• 41mMore Bang for Your Buck: How Yelp Autoscales Mesos + Marathon on AWS Spot FleetRob Johnson · MesosCon Europe 2017
• 42mAWS Spot instances: huge savings (or huge risks?)Taras Postument · code.talks 2019
• 30mKubernetes + Operator + PaaSTA = Flink @ YelpAntonio Verardi · Flink Forward Europe 2019
• 43mSpot InstancesDerek Felska · DevOps World | Jenkins World 2019 San Francisco
• 57mA Distributed and Fault-Tolerant Application with Apache Mesos & Apache AuroraFlorian Pfeiffer · FrOSCon 2015
• 29mGetting Started with Apache MesosRoger Ignazio · MesosCon Asia 2016
• 19mKeynote: Operating 600+ Mesos Servers on 7 Datacenters @CriteoPierre Cheynier · MesosCon Europe 2017
• 58mDistributed Scheduling with Apache Mesos in the CloudDiptanu Gon Choudhury · Philly ETE 2015