Grappling with the Bitter Lesson
summary
Argument: encoding bespoke organizational knowledge into rules loses to scaleable learning by LLMs. Uses: BERT then chat GPT prototype and a sandboxed assessment pipeline; method: model-to-model prompts, human review, mapping to 60,000-book/5,000-video catalog. Built: O'Reilly verifiable skills initiative — an on‑the‑fly skill‑framework generator that assesses users in the sandbox and flags content gaps; result: prototype gives accurate skill evaluations (correctly spots shell proficiency and an unexpected Azure dependency), while prompting business‑model and supplier‑royalty changes.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:44The bitter lesson in AI training
- 3:30O'Reilly as learning platform
- 6:23Make it new with AI
- 7:04Verifiable skills initiative
- 12:17AI skill assessment vs human
- 15:48Rethinking business models
- 18:34Adapting to AI challenges
- 21:41AI adoption stages
- 23:23Embracing the bitter lesson
related talks
21mChatGPT, LLMs, and LangChains - A Beginner's Guide
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI
01h 04mWhat are we thinking in the age of AI?
51mTowards a Pragmatic Understanding of Large Language Models
01h 04mLLM Metacognition: Understanding and leveraging "Thinking about Thinking"
45mPanel Talk
47mOperationalization of AI/ML models and use cases for ChatGPT Q&A
01h 04mBuilding Expertise Into Generated Conversations