CCEval: Accurately and Confidently Evaluating Performance Metrics of Congestion...
summary
CCEval is an evaluation framework that outputs confidence intervals and forecasts needed trials using model-free trial forecasting and speeds extreme-tail estimates with importance sampling + cross-entropy on top of standard simulators (NS3, M3, Parsimon) and a hardware dumbbell testbed for DCTCP, DCQCN, HPCC and Timely. It claims 1% relative error at 95% confidence level, reports required trials (often >25, rising toward 99.9th percentile), and cuts tail-trial counts by about 75–80% while matching simulator and testbed results.
AI-written summary · about
chapters (13)
- 0:00CCEval introduction
- 1:42Flaws in standard evaluation
- 3:31Variability and trial counts
- 4:34CCEval framework design
- 5:57Confidence intervals foundation
- 7:24Model-free estimation algorithm
- 8:25Limitations for tail quantiles
- 8:45Model-based tail quantile method
- 11:33CCEval evaluation results
- 13:20Trial count forecasting
- 14:49Traffic duration impact
- 15:53Model-based tail quantile benefits
- 16:35Conclusion and release
related talks
21mFast Confidence Estimation: Regression models
25mPCC Vivace: Online-Learning Congestion Control
21mModel-Agnostic and Efficient Exploration of Numerical State Space of Real-World...
17mTowards provably performant congestion control
17mPrecise Data Center Traffic Engineering with Constrained Hardware Resources
12mCATO: End-to-End Optimization of ML-Based Traffic Analysis Pipelines
17mBridging the Gap between QoE and QoS in Congestion Control: A Large-scale Mobile...
18mMeasuring Congestion in High Performance Datacenter Interconnects