Hold Me Closer, Tony Danza
summary
Real pipeline that converts any song into a multi‑track karaoke file: open‑source Demucs separates vocals/drums/bass, Whisper generates timed lyrics, an LLM (tested locally at ~14B and 30B sizes) corrects lyric errors while preserving timings, then FFmpeg packages stems and JSON lyrics into an MP4 stem container (new “car” atom). A web/Electron player and toolchain exist, visualizations use Butterchurn, processing runs on PyTorch, and a live demo shows the system working end‑to‑end.
AI-written summary · about
chapters (15)
- 0:00From Edison to Karaoke
- 1:06The CD+G Format and Its Limitations
- 3:00Reimagining Karaoke for the Web
- 4:05Why DJs Need Audio Stems
- 5:06AI-Powered Stem Separation with Demucs
- 6:04Using MP4 as a Multi-Track Audio Container
- 7:02Better Karaoke with Original Recordings
- 8:14Generating Timed Lyrics with Whisper
- 9:53Fixing Lyrics with LLMs
- 11:12Designing a Modern Karaoke File Format
- 12:18Building the Open Source Toolchain
- 15:01Live Demo: AI-Powered Karaoke
- 16:44In-Ear Monitoring, Coaching, and Future Ideas
- 18:21Building with AI as a Web Developer
- 20:11A Successor to CD+G
related talks
23mExploring the Potential of the Web Speech API in Karaoke
31mCats, The Musical! Algorithmic Song Meow-ification
30mCats, The Musical! Algorithmic Song Meow-ification
41mMusic and Machine Learning
32mCats, The Musical! Algorithmic Song Meow-ification
39mA Vue into Rock & Roll part 2
28mCats, The Musical! Algorithmic Song Meow-ification
55mAI Music Generation, From Prompt to Production