When the Only Way to Scale up, is to Scale Down
summary
Claims that less data often beats more data and shows tooling to prove it: Alectio uses active learning and an auto active learning optimizer to pick training subsets and avoid full retrains. Empirical notes: 40,000 zettabytes exist, 85% of data is dark or ROT, 60% loses value on collection, data centers cause ~2% of emissions trending toward 5.5% by 2025. On DeepWeeds experiments the best sample of 20 outperforms the worst sample of 80, and smart querying matches large-data accuracy while using far fewer labels.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:59About Myself
- 2:00About Alectio
- 3:37The History of Deep Learning & Big Data
- 5:36The Challenges of Big Data
- 9:41The Environmental Impact of Big Data
- 12:32Data Wastefulness
- 15:05There is Hope...
- 16:21Better with Less Data? Yes, it's possible...
- 18:49What is Active Learning?
- 20:42The Pros and Cons of Active Learning
- 22:00Let's Learn Active Learning (1/2)
- 29:44A Few Results
- 31:26A Better Breed of Active Learning
- 33:42Conclusions
related talks
17mFocus on Data, on the Road to AI
30mActive Learning and Human-in-the-Loop Machine Learning
37mMachine Learning, The Right Way
19mA world of data
49mPanel Discussion
26mThe scaling laws of AI Why neural networks continue to grow
18mUnleash your Private Datasets to Power up AI
30mEssential Ingredients in Scaling Organizations for ML