Paper Synthesizer! Music created with Augmented Reality
summary
Demonstrates a working Dynamic Medium / Augmented Reality rig that uses projection mapping onto a table and a webcam to read handwritten paper into sound, with concrete failures and fixes: OpenCV’s findContours to trace hand-drawn waveforms, SIFT and RANSAC for homography where possible, and a machine-readable tape workaround for repetitive patterns. Shows mapping of hand-drawn waveform outlines to X=time Y=amplitude plus HSV color detection for game-piece sequencing, end-to-end audio generation and a live performance with visible reliability limits.
AI-written summary · about
chapters (11)
- 0:00Paper Synthesizer project
- 1:34Hand-drawn waveform instruments
- 2:18Projecting onto physical objects
- 3:20Perspective geometry with linear algebra
- 4:47Homography and feature matching
- 5:56Go board pattern recognition
- 6:57Squiggles to waveforms
- 7:38Colored game pieces for control
- 8:20Debugging paper synthesizer demo
- 9:31Vision inconsistencies and redundancy
- 10:13Live demo and conclusion
related talks
03mStackable Music: A Marker-Based Augmented Reality Music Synthesis Game
35mMake Art Not Money
37mHow to code music?
24mMeet the artists
30mThe Formulartic Spectrum
31mBuilding a JavaScript Video Synthesizer for Live Performance
14mCombining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance
45mArt.js