Secure Numerical Computing is Hard: Lessons from 10 Years of Open Data Science & the Long Road Ahead
summary
Annual survey data plus 12 years and ~$30M of investment show open numerical stacks carry brittle, value-dependent correctness: Fortran/C extensions, JITs and exotic hardware make results depend on import order and chip features. Concrete failures include a GCC subnormal bug that turns tiny floats to zero, runtime hex-to-shared-library loading, and deleted binary symbols breaking downstream. Built response is Conda packaging to bundle native deps and enterprise visibility, but attacks multiply—data poisoning via typo-squatting and opaque training sets demand provenance and auditing.
AI-written summary · about
chapters (11)
- 0:00Introduction and speaker background
- 3:08What makes numerical computing unique
- 7:18Python’s role in numerical computing
- 10:49Challenges in Python packaging and dependencies
- 16:03Anaconda’s solution for open-source chaos
- 19:35AI’s foundation on a fragile stack
- 22:48Data poisoning and supply chain risks
- 27:19Transparency in AI training data
- 31:28Defining correctness in AI outputs
- 34:20Garbage in garbage out in AI
- 35:01Q&A and closing remarks
related talks
39mTrusting Your Data Science Pipelines: A Quantitative Approach
58mBuilding an Open Platform for Sustaining Data Science Innovation
45mDe Fortran a Julia: Computação Científica no Mundo Real
45mValues-based OSS Open Discussion
27mConsidering Rust for scientific software
34mData Science beasts (failures) and where to find them
01h 31mBig Data: The Computation/Statistics Interface
48mCommon Data Science Mistakes