Cognitive Biases in Performance Testing
summary
Claims that common cognitive biases cost five extra hours of outage by masking the true root cause, with tooling and numbers as evidence: Tricentis Flood cloud load tests hit 15 million users, logs queueing via Redis, DB connection leaks from containers, and an unexpected eight containers on one host. Surprising diagnostics: an NFR of 4.6 seconds and a 95th percentile of eight seconds hide long tails. The dashboard failed in real time; teams add a missing alert and a new graph and the platform resumes correct operation.
AI-written summary · about
chapters (17)
- 0:00Start
- 2:15Cognitive bias
- 8:50Anchoring effect
- 12:29Moving to percentiles
- 20:33Authority bias
- 22:56Confirmation bias
- 25:31Inattentional blindness
- 27:54Availability bias
- 28:56Identify which System of thought is appropriate for the situation.
- 30:45Use both systems of thought in tandem.
- 32:33Anchoring bias
- 33:00All measurements are wrong
- 34:03Frequency Distributions
- 34:28Heatmaps
- 36:25Inattentional bias
- 36:52Develop expert intuition.
- 37:06Seek independent sources.
related talks
08mCommon Mistakes in Performance testing
38mReal-Life Solutions for Test Environment Challenges
48mTesting The Cloud
33mThe Real Cloud Testing Story
15mWhy I Prefer Good Testing Over Excellent Testing
16mAre companies chasing a quality mirage in the era of cloud, devops, SaaS & others
10mBeyond “try harder”: Effective strategies to tackle bugs
47mHow to win in software development with tighter feedback loops