Attention is not all you need - Keynote
summary
Argues that LLMs are powerful but not universal, showing concrete alternatives: TP PFM pretrains transformers for spreadsheets, T-Rex and XLSTM return strong zero‑shot time‑series forecasting, Kronos 2 and Sundial beat rivals on multivariate and tokenized series, ConfExt V2 and DinoV3 give competitive vision results (with CNN distillation), All‑Tracker achieves real‑time optical flow, Malmo ACT and Gemini hit 45.1% on ARC, and a 7M‑parameter recursive model reaches 8% on ARC AGI2 — practical wins and surprising small‑model results.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:05Attention not sole AI solution
- 3:53Deep learning history
- 6:21AI beyond LLMs
- 8:30Spreadsheet transformers
- 10:16Time series with XLSTMs
- 15:13Vision models beyond ViT
- 19:27Neural architecture search
- 21:31Vector graphics with MLPs
- 22:39AI in medical domains
- 23:21AlphaGo and search
- 24:04World models and multimodal LLMs
- 25:30Efficiency limits
- 26:54ARC AGI challenge
- 28:19Math reasoning with neural nets
- 29:03Differentiable logic
- 30:51Evolutionary algorithms
- 32:16Information theory vs complexity
- 32:59Q&A
related talks
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence
54mKeynote: Can you trust your (large language) model?
44mKeynote: Tracking the Pulse of Generative AI
32mSponsored Session: Beyond the Buzz: Navigating AIs Myths and Realities
36mInsights from The State of AI Report 2023
12mKeynote: How LLMs Have Changed the Way Organizations Do AI
54mKeynote: Can you trust your (large language) model?
44mThe LLM Revolution: From Inception to the Future of Thinking Machines