Scott Clark, CEO & Co-Founder, SigOpt at MLconf Seattle 2017
summary
Claims Bayesian optimization using a Gaussian process surrogate and expected improvement acquisition beats grid and random search on real deep learning tasks, reaching better accuracy with roughly 10x faster wall‑clock/sample efficiency. Implementation is black‑box (continuous, integer, categorical), tested on an MXNet convolutional sentiment model over Rotten Tomatoes data via an AWS case study, uses open tools (MOE/Spearmint variants) with code and results published on GitHub and the AWS AI blog.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:32What is the most important unresolved problem in machine learning?
- 1:02SUNE MOTOR TESTER
- 1:18TUNABLE PARAMETERS IN DEEP LI
- 3:54STANDARD TUNING METHC
- 7:12OPTIMIZATION FEEDBACK LI
- 9:04OPTIMAL LEARNING
- 9:42BAYESIAN GLOBAL OPTIMIZA
- 10:36HOW DOES IT WORK?
- 12:33GAUSSIAN PROCESSES
- 16:48EXPECTED IMPROVEMENT
- 20:37STOCHASTIC GRADIENT DESC
- 21:00ARCHITECTURE PARAMETE
- 21:15TUNING METHODS
- 21:28SPEED UP #2: RANDOM/GRID -
- 21:51CONSISTENTLY BETTER AND FI
- 24:11TEXT CLASSIFICATION PIPEL
related talks
24mUsing Bayesian Optimization to Tune Machine Learning Models
41mOptimizing Machine Learning Models
24mScott Clark, Software Engineer, Yelp
18mAlexandra Johnson, Software Engineer, SigOpt, NYC 2017
28mBayesian Optimization- Fundamentals, Implementation, and Practice
34mBayesian Optimization and it's application to Neural Networks
39mBayesian optimisation with scikit-learn
03mDBTB INT Scott Clark r