Your Coflow has Many Flows: Sampling them for Fun and Speed
Talkat USENIX ATC '19 · on Friday, 2019-07-12· at 09:35· USENIX ATC Track II: Grand Ballroom VII–IX·
summary
A new scheduler called Philae samples a few flows per coflow (typically sampling 1% of flows or only for coflows with ≥7 flows), pre-schedules those samples at high priority to estimate average flow size, then orders coflows by Shortest Coflow First using size plus per‑port contention and weighted priority-queue shares to avoid starvation. Evaluated on Azure and Facebook traces, it cuts average CCT by 1.50× (P90 8.00×) on 150 nodes and 2.72× (P90 9.78×) at 900 nodes.
AI-written summary · about
chapters (21)
- 0:00Introduction
- 0:27Big Data Analytics
- 0:38MapReduce
- 0:51Communication Phase
- 1:26Coflow Abstraction
- 2:45Online Coflow Healing
- 2:58Proposed Online Coflow
- 3:33Outline
- 4:07Example
- 6:15Primary Drawbacks
- 7:02Roundrobin
- 9:01Recap
- 9:32Doubts about Sampling
- 12:20Practical Issues
- 13:49Valuation of Fillet
- 15:10Fillet Job Speed
- 16:16Fillet Sensitivity
- 16:37Summary
- 17:02Mario Agassi
- 18:52Practical Challenges
- 19:27Comparison with Coda
related talks
16mFlow Scheduling with Imprecise Knowledge
17mIs advance knowledge of flow sizes a plausible assumption?
17mPyrrha: Congestion-Root-Based Flow Control to Eliminate Head-of-Line Blocking in...
23mNetwork Flow Aggregation: Pay for the Logs You Care About!
16mNetwork Flow Data in the Cloud
21mTransparent and Flexible Network Management for Big Data Processing in the Cloud
21mTowards An Application Objective-Aware Network Interface
13mContracting Wide-area Network Topologies to Solve Flow Problems Quickly