Enhancing Performance Test Results with ML & LLMs: Communication Beyond the Numbers
summary
An automated pipeline turns raw K6 load tests into stakeholder-ready PDFs by cleaning anomalies with isolation forest, explaining feature importance and extrapolating with XGBoost, then generating deterministic summaries with an LLM and a Python/HTML PDF compiler. Concrete results: trained model shows users explain 25% of latency variance and predictions give a 95% CI around 4,000–5,000 ms for a scaled scenario. The system runs end-to-end and measurably improves nontechnical buy-in.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 4:23Communicating technical results
- 5:30AI-powered solution framework
- 7:41K6 performance testing
- 9:31Cleaning data with Isolation Forest
- 11:42Analyzing data with XGBoost
- 13:28Predicting unseen scenarios
- 15:14Metrics to narratives with LLMs
- 18:55Automated final report
- 20:45Old vs new reporting
- 22:10Pipeline recap and customization
- 23:35Q&A: Abnormal test results
- 24:39Q&A: Starting setup
- 26:04Q&A: Client conversations
- 27:10Q&A: Team collaboration
- 27:53Q&A: Recommended tools
- 28:57Closing remarks
related talks
28mML-Driven Test Intelligence at Scale - What Works What Fails and Why It Matters
39mMethodology of ML in Performance Testing (How to Find Bottlenecks Automatically)
10mML-based Log Analysis for Faster Debugging
14mImproving Large Language Model Performance
22mAccelerating ML impact
53mUnleashing the Black Box: Testing Machine Learning Systems
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
32mStrategies for LLM Evals (GuideLLM, lm-eval-harness, OpenAI Evals Workshop)