How the cloud changed the way I do analytics and research
summary
Uses h2o, AWS spot instances, and future + reticulate to scale experiments: spins up five cloud workers from a 2‑core laptop, runs gradient boosting and random forest on the Higgs dataset with five bootstraps and 5‑fold CV, returns logs and progress for each tree, and auto‑terminates instances. Claims 80–90% savings using spot pricing and shows a 64‑core / 0.5‑TB example costed roughly $400 for ~10 hours. The demo completes successfully and the workflow brings heavy analytics to cheap ephemeral machines.
AI-written summary · about
chapters (9)
- 0:00Introduction to cloud analytics
- 1:53Why switch from local tools
- 3:38Scaling machine learning models
- 5:03What is H2O framework
- 6:54Cloud computing benefits
- 8:43H2O model tuning advantages
- 11:15Parallel computing with AWS
- 13:49Live demo setup
- 19:50Q&A and summary
related talks
47mCloud Native and Sustainable, Reproducible Scientific Computing
36mAn almost perfect match: cloud native and scientific computing
36mMigrating Applications to the Cloud in a Cost Effective Manner
42mBuilding a green supercomputer in the cloud
22mRunning Geospatial Workloads on AWS
10mLightning Talk: Does Cloud Elasticity Pay off for HPC Workloads?
05mHadoop in the cloud: A Nielsen use case
50mScalable Data Science & Deep Learning with H2O