~/talk

CloudCluster: Unearthing the Functional Structure of a Cloud Service

on YouTube 15m
Talkat 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22) · 2022-04-04..04-06· [networking, data-engineering]

speakers

name:Weiwu Pang
links:none on file
name:Sourav Panda
links:none on file
name:Jehangir Amjad
bluesky:jehangiramjad.bsky.social
name:Christophe Diot
links:none on file
name:Ramesh Govindan
links:none on file

resources

video:https://youtu.be/DLdTMwORJis
slides:none on file

summary

CloudCluster extracts a service’s functional groups by clustering the VM-to-VM traffic matrix, using feature scaling, dimensionality reduction (via singular value decomposition), and agglomerative hierarchical clustering so no K is required. It scales to projects of thousands to tens of thousands of VMs, finds clusters that match ground truth (over 92% homogeneity and 94% completeness), detects 25 real structural anomalies (all detected), and enables operational actions such as a reported 40% cost reduction for one functional group.

AI-written summary · about

chapters (24)

  1. 0:00Intro
  2. 0:54Anatomy of a Cloud Service: The Customer View
  3. 1:24Anatomy of a Cloud Service: The Provider View
  4. 2:35Today: Monitoring at Different levels of Abstract
  5. 3:00A New Abstraction: The Functional Structure
  6. 3:48What do we mean by Functional Structure?
  7. 4:39The Provider Does Not Know the Functional Stru
  8. 4:56Extracting the Functional Structure
  9. 5:29Hypothesis
  10. 6:28Approach: Use Clustering!
  11. 6:43Cloud Cluster: Challenges
  12. 7:17The Problem
  13. 7:48Insight: Exploit Redundancy in VM-to-VM Traffic
  14. 8:09Dimensionality Reduction
  15. 8:46Agglomerative Clustering
  16. 9:15Generating the Clusters: Where to cut the dendre
  17. 9:28Use Cluster Inconsistency to Decide!
  18. 9:43Evaluating Clustering Performance
  19. 10:28Cloud Cluster. Excellent Agreement with Ground
  20. 11:32What is the Functional Structure Useful For?
  21. 12:06Detecting Traffic Shifts with Cloud Cluster
  22. 12:38Detecting Structural Changes
  23. 13:20Anomaly Detection
  24. 14:04Summary

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.