Taking Control of LLM Outputs: An Introductory Journey into Logits
summary
Shows how raw logits from LLAMA-3B-instruct drive deterministic token scores and how direct logit manipulation—token masking via constraint decoding—forces valid JSON and regex-constrained outputs, speeding generation by skipping predictable tokens. Demonstrates greedy, temperature, top‑K/top‑P, and beam search (example beam width 2), introduces Outlines and Guidance for local structured decoding (token‑healing on Guidance only for local models), and repurposes per-token logit loss as a “surprise” evaluation metric.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:07LLM generation basics
- 2:22Understanding logits
- 6:10Sampling strategies
- 11:58Uncertainty detection
- 14:22Structured generation
- 18:24Libraries for control
- 22:23Advanced features
- 23:56Q&A and conclusion
related talks
13mGetting LLMs to do what you want: Output controllers
15mKeynote: Into the Black Box: Observability in the Age of LLMs
01h 49mAI engineering with open access LLMs that lie, curse, and steal
22mSmarter word generation in LLMs: Guiding sampling through output probability distributions
35mReading the Mind of an LLM
02h 52mLow Level Technicals of LLMs
28mExplaining Machine Learning Models with LLMs
39mHallucination-Free LLMs: Strategies for Monitoring and Mitigation