Having fun with Music and Keras
summary
Builds a simple NumPy synthesizer and an infinite synthetic dataset of notes, chords and drums, then trains a Keras model that embeds an FFT inside the network. Claims the FFT-branch Keras model can separate a melody line from mixes; after a few epochs it reconstructs guitar well on synthetic data and reasonably on a full-band recording (drums still leak). Provides implementation details, examples of synthesis methods (additive, subtractive, envelopes) and concrete quantitative setup choices like 44.1 kHz sampling and short-window training slices.
AI-written summary · about
chapters (19)
- 0:00Music and deep learning
- 1:25Simple synthesizer with NumPy
- 4:36Audio data and waveforms
- 8:52Single tone and spectrogram
- 11:02Harmonics and additive synthesis
- 13:12Envelopes and decay
- 14:14Kick and snare drums
- 14:59Subtractive synthesis
- 16:24Advanced synthesis techniques
- 16:45Music theory and notes
- 18:31Chords and improvisation
- 19:12Music with random walks
- 21:33Melody from multi-instrument tracks
- 23:19Dense model for melody extraction
- 25:39FFT for melody detection
- 27:06Evaluating FFT model
- 28:21Testing on real music
- 29:37Model on synthesized music
- 30:22Key takeaways
related talks
43mMaking machines that make music
21mMusical Deep Neural Networks in the Browser
24mCreating an AI Musician with JavaScript
19mDo Repeat Yourself
03mBuild a Synthesizer with Python
20mSoftware-defined declarative synthesizer live-coding in a jupyter notebook
31mHow to Use Machine Learning and .NET to Write Crushing Metal Riffs
34mModelling emotional nuance of musical performance