Doug Eck, Research Scientist, Google
summary
Magenta experiments ship concrete code and demos: QuickDraw (~1B sketches) feeds a sketch-rnn variational autoencoder that models strokes as Δx, Δy plus pen-up, trained on 20-second drawings and sampled unconditionally via a Gaussian latent Z (avoids memorization). A stroke decoder uses an RNN with a mixture of Gaussians for XY. On music, models train on Disklavier performance dataset (real pianist timing and velocity), produce evocative unconditional LSTM samples but lack long-range form, so a measure-level hierarchical VAE with separate decoders for drums/bass/lead is developed and yields plausible interpolations between measures.
AI-written summary · about
chapters (6)
- 0:00Start
- 11:04Quick-Draw
- 13:56Hot Yoga Is More Dangerous than Cold Yoga
- 15:49The Variational Model
- 19:33Closing Thought
- 23:04Unconditional Samples
related talks
47mExpressive AI-Generated Music With Google's Performance RNN
46mMusic Generation with Magenta: Using Machine Learning in
01h 04mKeynote 3. Interactive generative models for music
28mMusic and AI in the Browser with TensorFlow js and Magenta js
38mMaking Art with Artificial Intelligence: Artists in Conversation
47mUnleashing your musical creativity with Magenta.js
21mMusical Deep Neural Networks in the Browser
41mMusic and Machine Learning