Speech Synthesis with Tacotron2 and PyTorch
summary
Implements Tacotron2 with PyTorch and trains on mel spectrograms to map text to audio samples, using an English public dataset of ~24 hours and a selfmade German corpus created semi‑automatically with audiotok and Speaker diarisation. Shows audio demos and concrete failure modes: multi‑speaker contamination breaks alignment, transcription noise from Google Cloud harms training, and intelligible output appears after tens of hours of training (48–96h) but quality depends on more homogeneous data and vocoder tuning.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:04Art to deep learning
- 3:42Style transfer with comics
- 7:10Hirschfeld and radio dramas
- 7:53Speech synthesis focus
- 10:26Audio representations
- 15:23Tacotron2 concept
- 17:49PyTorch advantages
- 20:36PyTorch resources
- 22:02Hardware setup
- 23:29Training on English data
- 25:17German synthesis challenges
- 26:47Data acquisition process
- 29:39Data preparation
- 30:45Speaker selection for synthesis
- 32:32Results and optimization
- 33:14Future directions
related talks
22mBuilding next-gen speech synthesis for Bookmate audiobook service
40mHow I Made My Computer Write it's First Short Story
22mVocalPy: a core Python package for acoustic comms research
25mBeyond Transcription: Building Voice AI That Understands Conversations
49mMartin Förtsch, Thomas Endres & Jonas Mayer
43mTalk by Om Thakkar (OpenAI)
26mMake Python Talk, Make Python Listen
22mWhy TTS Models Now Look Like LLMs