~/talk

ServiceLab: Preventing Tiny Performance Regressions at Hyperscale through...

on YouTube 16m
Talkat 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24) · 2024-07-10..07-12· [devops-and-delivery, testing]

speakers

name:Mike Chow
links:none on file
name:Yang Wang
links:none on file
name:William Wang
x:kleptographic
name:Ayichew Hailu
links:none on file
name:Rohan Bopardikar
links:none on file
name:Bin Zhang
links:none on file
name:Jialiang Qu
links:none on file
name:David Meisner
linkedin:davidmeisner
name:Santosh Sonawane
links:none on file
name:Yunqi Zhang
linkedin:~yunqi
name:Rodrigo Paim
links:none on file
name:Mack Ward
links:none on file
name:Ivor Huang
links:none on file
name:Matt McNally
links:none on file
name:Daniel Hodges
links:none on file
name:Zoltan Farkas
links:none on file
name:Caner Gocmen
links:none on file
name:Elvis Huang
links:none on file
name:Chunqiang Tang
links:none on file

resources

video:https://youtu.be/pMqG-3bIw9A
slides:none on file

summary

ServiceLab is a pre-production A/B performance platform that replays sampled RPCs across on-demand heterogeneous machines to catch tiny regressions before rollouts, detecting regressions as small as 0.01% and reporting a 0.14% median regression on a serverless front-end running on >500k machines. A study of 21M hardware benchmarks finds five factors account for 87% variance (server type, CPU, kernel, turbo, datacenter), Trials multiply parallel replicas for tighter CIs, and static analysis selects 23% of commits to test, with 3% signaled.

AI-written summary · about

chapters (12)

  1. 0:00Introduction
  2. 0:15Challenges at hyperscale
  3. 3:57ServiceLab overview
  4. 6:07Experiment design
  5. 7:54Dependency management
  6. 8:53Allocating machines
  7. 10:02Detecting tiny regressions
  8. 11:05Adaptive statistical methods
  9. 12:30Production experience
  10. 13:03FrontFaaS case study
  11. 13:42Results and feedback
  12. 13:56Key takeaways and conclusion

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.