Alexandra Johnson, Software Engineer, SigOpt, NYC 2017
summary
Hyperparameter tuning uses SigOpt Bayesian optimization to search real model settings not defaults, yielding concrete wins like a 315% boost in accuracy on a TensorFlow recommender; baseline XGBoost digit classifier jumps from ~70% to much higher when feature-extraction params join model params. Emphasizes cross‑validation to avoid overfitting, warns about metric misalignment (Microsoft search AB test) and competing objectives, and contrasts methods: grid search fails from the curse of dimensionality, random search has high variance, while Bayesian suggesters exploit past trials.
AI-written summary · about
chapters (10)
- 0:00Intro
- 0:13What are Hyperparameters?
- 3:16Trusting the Defaults
- 5:00Using the Wrong Metric
- 7:36Overfitting
- 9:08Too Few Hyperparameters
- 12:08Hand Tuning
- 13:54Grid Search
- 15:42Random Search
- 17:52Thank You!
related talks
41mOptimizing Machine Learning Models
24mUsing Bayesian Optimization to Tune Machine Learning Models
18mBest Practices for Hyperparameter Optimization
26mAddressing Some of Challenges When Optimizing Long-to-Train-Models
03mDBTB INT Scott Clark r
26mScott Clark, CEO & Co-Founder, SigOpt at MLconf Seattle 2017
34mBayesian Optimization and it's application to Neural Networks
26mModel Selection with Python: An Introduction to Hyper Parameter Tuning