AI Intuition: Exploring Language Model Latent Space
summary
Argues that language-model “intuition” maps to sudden reorganization in latent geometry, cites gamma bursts and induction heads forming during training, and links MLPs to topology-squeezing and the superposition hypothesis. Demonstrates inspection tools: gemoscope/Neuropedia for activations, token-explorer and loom for path tracing, and a proof-of-concept called prompt robber that reconstructs concept networks between input and output — it kind of works and is open for contribution. Provides prompting strategies grounded in steering latent directions rather than verbose labels.
AI-written summary · about
chapters (11)
- 0:00Introduction and background
- 4:32Defining intuition for humans
- 8:35Understanding language model latent space
- 11:07Text embeddings and contextual meaning
- 15:51Attention mechanisms in transformers
- 18:26Multilayer perceptrons and topology
- 21:40Neurons and poly-semantic features
- 24:10Mechanistic interpretability
- 26:04Exploring latent space with tools
- 32:59Guidelines for navigating latent space
- 41:14Q&A session
related talks
59mBetween the Layers– Interpreting Large Language Models
01h 01mBetween the Layers– Interpreting Large Language Models
40mIs my Large Language Model a Strange Loop?
52mGeneralization in the representations and computations of frontier language models.
35mReading the Mind of an LLM
51mTowards a Pragmatic Understanding of Large Language Models
41mSub-Linear Algorithms meets Large Language Models
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence