How I Made My Computer Write it's First Short Story
summary
Uses PyTorch and a local eGPU / GTX 1080 to train models that attempt full episodes: character-aware text generation from fan transcripts, TTS trained on the LJ Speech corpus (30k clips, ~24 hours) and audio alignment via Google Speech API plus n‑gram matching. A German corpus of ~1.5 million words / 56k unique words proves too small for clean dialogue generation; character‑based RNNs give amusing output, Tacotron‑style TTS produces intelligible lines after days of training, and image style transfer reproduces plausible covers but voice style transfer remains unfinished.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:09Neural networks in art
- 3:19External GPU setup
- 5:06Neural style transfer with comics
- 8:17Frameworks and setup hurdles
- 10:02Style transfer to radio dramas
- 11:50Target series format
- 13:58Text data preprocessing
- 16:51Text generation with RNNs
- 20:58Dialogue generation with chatbots
- 22:02Speech synthesis with neural networks
- 26:55Training speech synthesis
- 27:58German speech synthesis challenges
- 28:44Separating dialogue from audio
- 30:53Artwork matching series style
- 33:25Evaluating generated artwork
- 34:33Key takeaways and future work
- 35:17Local meetups and events
- 36:02Q&A
related talks
35mSpeech Synthesis with Tacotron2 and PyTorch
25mGenerative Text Modelling: Scratching the surface
22mBuilding next-gen speech synthesis for Bookmate audiobook service
41mGPU Training Made Easy-Pre-train & fine-tune NLP models
22mWhy TTS Models Now Look Like LLMs
26mMake Python Talk, Make Python Listen
44mHow to build your own Tiny Language Model from scratch
49mMartin Förtsch, Thomas Endres & Jonas Mayer