Context-Aware Masking for Controlled Music Generation in MIDI
summary
Preprocessing and masked-infilling methods target MIDI performance data with a 10 millisecond sampling grid, tokenizing velocity, note_on/time increments and note_off events so GPT-style models ingest performance nuance. A small proof shows a 300 million parameter GPT produces plausible continuations; larger training uses an internal corpus of 31 million notes (≈200M musical tokens) and 4B combined tokens. The pipeline includes Hugging Face datasets, foran/streamme visualizations and a pianist crowdsourcing site, enabling composer token style transfer and controllable above/below-median infilling that works in examples.
AI-written summary · about
chapters (10)
- 0:00Piano GPT project intro
- 2:06Piano and MIDI basics
- 7:39GPT music generation
- 10:08Control tokens for music
- 12:13Masked language modeling
- 15:04Real piano performance data
- 19:33Tokenizing music data
- 22:27Pre-training GPT on music
- 25:15Fine-tuning for control
- 31:15Summary and tools
related talks
34mModelling emotional nuance of musical performance
58mMusic Generation via Masked Acoustic Token Modeling
01h 04mKeynote 3. Interactive generative models for music
41mInside GPT – The Maths Behind the Magic
05mInteractive Exploration-Exploitation Balancing for Generative Melody Composition
43mMaking machines that make music
41mMusic and Machine Learning
47mExpressive AI-Generated Music With Google's Performance RNN