The causality trap: why your LLM sounds smart but thinks shallow
summary
LLMs pattern-match associations but cannot compute interventions: benchmark shows ~83% accuracy on associations but near-random on interventional/rung‑2 questions. Using Kaggle Amcard coupon data, an R workflow defines a DAG, runs do-search to get adjustment sets, and estimates effects with linear regression — drugstore coupons show ~+$30 effect after adjustment while ready‑to‑eat coupons drop to ~0. An LLM (Claude) returns association-based answers and misleads; combining LLM-assisted DAG discovery with deterministic statistical engines produces auditable, actionable causal estimates.
AI-written summary · about
chapters (8)
- 0:00Causal parrots and LLMs
- 3:49Causal inference in business
- 16:08LLMs fail at causal reasoning
- 20:20Benchmarking LLM limits
- 20:43Live demo: causal workflow
- 27:50LLMs scale causal inference
- 31:24Pearl's ladder role
- 32:28Q&A: Sample size
related talks
27mExploring the Mechanisms of LLM Reasoning Capabilities
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
31mBeyond LLMs and Maturing RAG use cases
52mGive Your LLMs a Left Brain
47mGive Your LLMs a Left Brain
52mAnatomy of an LLM
32mEvaluating probabilistic decision making by LLMs: a decision-theoretic framework and application...
01h 29mLLM Safety, Alignment, and Generalization