Controllable and Creative Natural Language Generation
summary
Argues that creativity requires constraint-aware search and shows three concrete results: lyric generation (Lyra) uses a hierarchical plan plus a modified beam search enforcing segmentation constraint and rhythm alignment to a melody; a distilled HMM guides decoding to satisfy regular-string constraints in a method called Control-G (2B HMM distilled from a 7B LLM, outperforming larger models on insertion/controllable edits); and human evals reveal models overuse a “rags‑to‑riches” arc (~80–90% vs ~30% human), exposing metric gaps.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 7:45Creativity needs constraints
- 14:29Lyric generation with melody
- 28:00Controllable HMM generation
- 36:56Evaluating narrative creativity
- 43:08Creativity metrics limits
- 52:06Future of creative AI
- 53:12Q&A
related talks
01h 18mThe Enigma of LLMs: on Creativity, Compositionality, Pluralism, and Paradoxes
26mTowards controllable language generation
01h 00mHuman and machine inductive biases for compositional linguistic generalization
43mComedy Writing With Small Generative Models
29mBeyond Solo: Enhancing LLMs through Collaborative Dialogue with GAN-Inspired Approaches
17mHallucinating Zombies, Creative Geniuses Exploring AI's Risky and Revolutionary Side-
53mGenerative AI: Overview and use cases
01h 39mMastering LLMs: From Prompt Engineering to Agentic AI