Automated Machine Learning & Tuning with FLAML
summary
Library demo shows actual wins not promises: R^2 ~0.7→0.8 on a regression, a few-percentage-point classification lift (66.7→67.3) and AUCs reported up to ~0.81 after tuning; best configs include concrete hyperparameters (e.g. LightGBM with hundreds of trees). Two APIs exist: task-oriented AutoML and a generic tune(fn) for arbitrary functions. Spark support runs parallel trials; LLM tuning uses an inference-budget aware search to pick model, temperature, n and token limits and returns cost-per-sample estimates in cents.
AI-written summary · about
chapters (13)
- 0:00Intro to Automated ML with FLAML
- 5:06What is FLAML
- 10:21FLAML architecture
- 18:29AutoML workflow
- 25:33Custom models and metrics
- 31:39Tuning user functions
- 41:30Advanced optimization algorithms
- 44:02Efficient tuning constraints
- 48:11Parallelizing with Spark
- 57:07Cost-effective LLM tuning
- 1:07:13Optimizing LLM prompts
- 1:12:17Tuning LLMs for math
- 1:18:15FLAML community & future
related talks
35mCreating Your Own LLM Tuning Platform with Open Source Technologies
16mAmeet Talwalkar, Assistant Professor of Computer Science, UCLA @ MLconf SF Part 2
10mAuto-tuning vllm
07mAmeet Talwalkar, Assistant Professor of Computer Science, UCLA @ MLconf SF Part 1
22mMulti-level Machine Learning-Guided Autotuning for Efficient Code Generation on a Deep(…)
38mFantastic ML apps and how to build them
33mML: Automated Machine Learning
39mMachine Learning with Scala on Spark