Braxton McKee, CEO & Founder, Ufora
summary
An automatically distributed, implicitly parallel Python platform that cross‑compiles a restricted, purely functional subset of Python and runs on AWS spot instances. It claims 300x speedups on a prime‑test demo (an hour to ~10s) by spawning threads across 120 cores and scaling to thousands, migrating stack frames across machines and learning data layout to colocate hot blocks. It keeps state on workers, is deterministic, and is open‑source under Apache 2.0 with core Python 0.1 and upcoming pandas/numpy reimplementations.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:03Tools fail at scale
- 2:25Fast distributed Python
- 3:28Open-sourcing Ufora
- 4:10Julia Spark vs pandas
- 5:35Ufora in practice
- 7:01Prime factorization demo
- 8:25Pure functional code
- 9:47Architecture and retention
- 10:50Parallel stack frames
- 12:37Cache optimization
- 14:22Non-contiguous data
- 16:28Performance optimization
- 17:10Python transition
- 18:34Closing
- 19:02Q&A
related talks
59mPython Meets Heterogeneous Computing
29mBlazing-Fast Python in Your Database: Unlocking Data Science at Scale with Exasol
01h 01mParallel Python Analyzing Large Data Sets
01h 06mBodo: Automatic HPC Performance and Scaling for Data Processing in Python
01h 10mJAX: accelerated machine learning research via composable function transformations in Python
27mSpeed up Python with hardware accelerators
47mMegacore: Megafast, Megacool?
55mAccelerated Python: The Community and Ecosystem