Bayesian Optimization and it's application to Neural Networks
summary
Claims that Bayesian optimization finds better hyperparameters for neural nets with far fewer costly trainings than grid or random search, modeling the outer objective with Gaussian Processes and choosing points by an expected improvement acquisition function. Examples use keras networks and the Python tools hyperas/hyperopt on datasets like MNIST, showing measurable accuracy gains and concrete discoveries (e.g., larger hidden layers drive higher dropout) rather than vague promises or demos.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:28Neural network training
- 5:24Hyperparameter challenges
- 8:35Manual vs random search limits
- 11:07Automatic tuning need
- 13:38Bayesian Optimization basics
- 17:33Gaussian processes in BO
- 21:11Computational costs
- 23:48Experiment insights
- 24:53Python BO libraries
- 25:41Q&A
related talks
28mBayesian Optimization- Fundamentals, Implementation, and Practice
24mUsing Bayesian Optimization to Tune Machine Learning Models
48mBayesian Optimization of Gaussian Processes with Applications to Performance Tuning
41mOptimizing Machine Learning Models
39mBayesian optimisation with scikit-learn
29mBayesian optimization with Scikit-Optimize
18mBest Practices for Hyperparameter Optimization
18mAlexandra Johnson, Software Engineer, SigOpt, NYC 2017