Follow the gradient: an introduction to mathematical optimisation
summary
Workshop explains which optimization problems are easy, which are hard, and shows working code: linear and convex quadratic formulations solved with CVXpy, quantile and elastic‑net regressions translated to LP/QP, a diet LP that yields a 71pence optimal breakfast, a knapsack integer program that times at ~60 seconds for 100k items and fails at a million, and a tiny neural net trained with autograd on MNIST reaching ~96% accuracy. Notebooks and Binder reproduce each result.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:11Optimization in machine learning
- 3:21Optimization problems
- 12:54Linear programming basics
- 17:13Diet optimization example
- 25:54Knapsack problem
- 29:27Quantile regression
- 38:39Convex programming
- 43:41Elastic net regression
- 51:05Follow the gradient
- 1:07:49Newton's method
- 1:14:57Stochastic gradient descent
- 1:17:02Neural network backpropagation
- 1:21:37Summary
related talks
39mPractical Optimization for Stats Nerds
01h 18mOptimization I
01h 15mEducational framework for Black Box optimization methods design
33mA Hands-On Introduction To Multi-Objective Optimization
19mML ain’t your only hammer: adding mathematical optimisation to the data scientist’s toolbox
01h 10mAnalysis and Design of Optimization Algorithms via Integral Quadratic Constraints
42mOptimization Crash Course
46mSmooth Nonconvex Min-Max Optimization