Questions for Theory in the New Age of Machine Learning
summary
Argues that LLMs become the learning substrate: using GPT-4 to generate justifications from 125 examples of CK-12 hints, distilling them into seven principles rubric that boosts classification without numeric parameter tuning; builds a running agent called Predict Anything that searches web sources (FRED, Google Trends), writes Python data-fetching code, logs per-function traces for Claude to self-reflect, and iteratively improves tasks; reports an unemployment prediction moving 4.30% → 4.23% and finds school-calendar signals improve flu forecasts.
AI-written summary · about
chapters (8)
- 0:00Workshop introduction
- 3:56Conventional ML wisdom
- 7:12Challenging ML assumptions
- 10:32LLM justifications for classifiers
- 18:11Predicting without features
- 24:14Agents writing their own code
- 31:48Theory opportunities in ML
- 33:36Audience discussion and reflections
related talks
17mPsychology of AI
24mHarnessing the Power of LLMs in Advanced Analytics
46mPlaying the long game
37mGenerative AI: Beyond (Just) Hype
41mThe Value of LLMs - Why Stupid Machines Are Smarter Than You Think
01h 02mA hacker's guide to open source LLMs
46mBeyond Basic Prompting: Supercharging Open Source LLMs with LMQL's Structured Generation
54mKeynote: Can you trust your (large language) model?
