tidymodels: Adventures in Rewriting a Modeling Pipeline
summary
Rewrites an in-production R package into a tidymodels pipeline, replacing hand-rolled dplyr/tidyr preprocessing and custom Stan tuning with recipes, parsnip, tune, broom, workflows and rstanarm where practical. Claims a measurable speed gain (a 30 minute end-to-end run becomes ~60 seconds, a 30× speedup), easier feature selection, safer preprocessing to avoid leakage, simpler hyperparameter tuning, automated production handoffs, and fast what-if simulations for stakeholders — all delivered while preserving or improving model fits.
AI-written summary · about
chapters (17)
- 0:00Introduction to tidymodels
- 1:09Need for pipeline rewrite
- 2:57Discovering tidymodels
- 4:04Recipes for preprocessing
- 5:09Parsnip model spec
- 6:13Tune hyperparameters
- 6:56Workflow integration
- 7:59Unexpected benefits
- 9:00Custom LEGO steps
- 10:25Post-processing insights
- 11:07Speed improvements
- 12:52Transition challenges
- 13:59Live product rewrite
- 14:40Real-time stakeholder updates
- 15:43Faster updates engagement
- 16:05Automation deployment
- 16:25What-if scenario modeling
related talks
57mGood Practices for Applied Machine Learning- 57mGood Practices for Applied Machine Learning
22mModelling, Tidyverse Way
15mThe tidysynthesis R package
21mTidypredict with recipes, turn workflow to SQL, spark, duckdb and beyond
19mtidymodels for time-to-event data
23mTotal Tidy Tuning Techniques
23mparsnip A tidy model interface