How Do We Rank Project Risk?
summary
Proposal: rank open source projects by expected dollar exposure computed from separately elicited event frequency and event magnitude estimates, multiply their probability distributions via Monte Carlo (1,000,000 samples, 100 bins) using triangular distributions, then aggregate multiple expert estimates by the mean of modes to produce a sortable risk score. Demonstrated with a working SEER prototype (not production), addresses sparsity across millions of projects and highlights obscure high‑risk “lib nebraska” packages; cites $1 trillion annual cybercrime scale as a motivating figure.
AI-written summary · about
chapters (17)
- 0:00Start
- 1:10Agenda
- 1:53What We Mean by Risk
- 2:17Event Magnitude
- 4:55The Criticality Score
- 6:14Context Problem
- 9:25Options for Eliciting Expert Opinions
- 13:54Plurality Voting
- 17:19Score Voting
- 19:18How Good Are the Estimates
- 19:38Calibration
- 23:48Demo
- 24:41What Is an Estimate
- 25:22The Decomposition of Risk into Magnitude and Frequency
- 30:35Estimate Values
- 32:58Possibility Plot
- 37:36When and Why Should We Re-Estimate a Project
related talks
45mThe Business Value of Security
43mChanging from Probability to Prioritization Method in Risk Measurement
25mOWASP Top 10 Risks for Open Source Software
24mNavigating Open Source Project Risk
51mFAIR STRIDE - Building Business Relevant Threat Models
44mOpening Keynote: Prove It! Confronting Security With Data.
58mNothing to Risk but Risk Itself: Expanding Vulnerability Risk with Internet-Scale Data
26mSimple Method of Automatic Risks Assessment for Web Systems Considering Assets Sensitivity