Music and AI in the Browser with TensorFlow js and Magenta js
summary
Shows runnable browser demos that use TensorFlow.js and Magenta.js to generate music: an arpeggiator driven by Improv RNN, a drum machine from drums RNN, and multi‑track loops from a MusicVAE latent space. Models load from Google CDN, run on GPU in the page, and output MIDI‑like note arrays played with Tone.js/Web Audio. Explores a 256‑dim latent space visualized as a 7x7 grid, performs 5–8 step interpolations, and maps PoseNet hand positions (150 ms sampling) to select and tempo drum patterns.
AI-written summary · about
chapters (9)
- 0:00Introduction to music AI
- 1:22Abstractions over neural networks
- 3:09Demo of arpeggiator synthesizer
- 8:46Generating and playing melodies
- 11:38Drum machine with neural nets
- 12:20Interactive music generation
- 15:35Music latent space exploration
- 20:34Visual control of drum patterns
- 27:05Conclusion and resources
related talks
21mMusical Deep Neural Networks in the Browser
47mUnleashing your musical creativity with Magenta.js
24mCreating an AI Musician with JavaScript
36mMusic and Art with ML
41mMusic and Machine Learning
46mMusic Generation with Magenta: Using Machine Learning in
39mMachine Learning Magic for Your JavaScript Application
47mExpressive AI-Generated Music With Google's Performance RNN