Introduction to Music Information Retrieval using Essentia.js
summary
Essentia.js ports native Essentia to Essentia.js via WebAssembly and exposes ~200 algorithms (2.7 MB build) plus a TypeScript API; it runs mel spectrograms, pitch/chord detection and onset tracking in the browser, uses audio worklet + worker for real‑time frames and a ring buffer for 512/128 sample handling, and runs Tiny TF.js models (typical web models ~2–3 MB, one VGG‑like model 62M params / 200+ MB). Custom C++ extractors handle missing or faster pipelines (used to detect saturation in a distributor QC demo), and the demos work end‑to‑end.
AI-written summary · about
chapters (18)
- 0:00Introduction to Essentia.js
- 1:07Presenting the team
- 3:18What is Music Information Retrieval?
- 17:53Essentia library overview
- 21:25Why Essentia.js was created
- 31:50Live demos with Essentia.js
- 40:16Hello World with Essentia.js
- 45:56Using workers for audio processing
- 51:23Script processor node real-time demo
- 1:01:17Chord detection in real-time
- 1:22:05Audio worklets with Essentia.js
- 1:25:56Machine learning on the web
- 1:30:57Essentia.js models for ML
- 1:42:13Real-time auto-tagging demo
- 1:53:06Code walkthrough for ML models
- 2:01:19Real-time ML inference setup
- 2:16:52Industrial use case at Sonosuite
- 2:26:39Custom extractors and Q&A
