The nuts and bolts of music source separation
summary
An in-depth run-through of music source separation explains that audio has 44,000 samples per second making datasets terabytes large, and separation must generate entire audio tracks from one mix. Methods use Fourier transforms to make spectrograms and rely on continuous masks rather than binary segmentation; complementary waveform models tackle raw sequences. Practical choices include using MobileNet to run experiments in hours instead of days, and exploring Transformers (used in Whisper) which so far are not yet applied successfully to music separation.
AI-written summary · about
chapters (14)
- 0:00Intro
- 1:33Title
- 2:03Agenda
- 3:00Music source operation
- 4:32Multidisciplinary approach
- 5:30Modeling audio data
- 6:06Masking
- 6:43Waveform models
- 7:18The best approach
- 7:43Practical tips
- 8:56Stateoftheart methods
- 9:53MobileNet
- 11:23What can be done
- 12:46Digital signal processing
related talks
25mLooking at Sound
45mAuditory Intelligence : beyond speech recognition
51mThe Merger of Machine Learning, Music and Programming
30mSound Modelling - parametric methods and deep learning...
58mMusic Generation via Masked Acoustic Token Modeling
03mOpen.Unmix.App – Towards Audio Separation On The Edge
38mThe Magic of Music Matching
31mHaving fun with Music and Keras