Is advance knowledge of flow sizes a plausible assumption?
summary
Paper evaluates whether flow sizes can be known ahead of time and finds complete clairvoyance likely intractable but partial knowledge is usable. It builds estimators using heuristics, system calls, gradient boosting trees and neural nets trained on synthetic and university cluster traces, evaluates them in pFabric and coflow scheduler SynChrony and reports small MFT loss versus perfect knowledge but high sensitivity for coflows. Measurement highlights include 5 microseconds on CPU, 1.3 microseconds on FPGA inference latency and up to 3x slowdown with zero knowledge.
AI-written summary · about
chapters (6)
- 0:00The importance of flow size
- 1:06Flow scheduling paradigms
- 3:46Acquiring flow knowledge
- 8:00Evaluating performance impact
- 10:22Knowledge versus effort
- 14:08Conclusion and Q&A
related talks
16mFlow Scheduling with Imprecise Knowledge
21mYour Coflow has Many Flows: Sampling them for Fun and Speed
21mTransparent and Flexible Network Management for Big Data Processing in the Cloud
14mCrystalPerf: Learning to Characterize the Performance of Dataflow Computation...
16mCredence: Augmenting Datacenter Switch Buffer Sharing with ML Predictions
40mOptimizing AI/ML Workloads
21mTowards An Application Objective-Aware Network Interface
17mDefeating Slow-and-Low Threats via Diffusion Model-based Generative Inference