Understanding and Steering Generative AI Systems
Talkat Special Year on Large Language Models and Transformers, Part 1 Boot Camp · 2024-09-04..09-06·
summary
A practical pipeline decomposes CLIP into patches, heads and neurons and shows empirically that mean ablation removing everything but the last ~4 attention layers leaves performance intact; heads get a text-aligned basis (about 30,000 text directions generated with GPT‑4 assistance) and a ~70‑dim subspace, neurons are approximated as rank‑1 functions (≈50% variance) then sparsely coded. Results give zero‑shot segmentation that outperforms prior methods, enable targeted head surgery to fix spurious cues (waterbirds), and expose surprising failures (counting limits, piano→violin via tulips, frog→turkey) useful for debugging.
AI-written summary · about
chapters (11)
- 0:00Introduction to interpretability
- 1:45Background on CLIP
- 6:33Transformer architecture basics
- 10:08Decomposing model layers
- 15:55Attention contribution analysis
- 20:22Attribution and segmentation
- 22:59Interpreting attention heads
- 33:00Surgery on spurious cues
- 37:27The counting mystery
- 44:45Neuron-level analysis
- 50:54Conclusion and future work
related talks
26mGenerative AI: Get a Grip … and a Look Under the Hood
45mHow Generative AI Works: LLMs and RAG Under the Hood
19mLet's Talk GenAI
46mAttention to Detail: Fine-Grained Vision-Language Alignment
01h 01mBetween the Layers– Interpreting Large Language Models
32mHow generative AI is Changing the way people interact with computers
53mGenerative AI: Overview and use cases
15mHarnessing the Power of Generative AI for Intelligent Applications