Pictures with Sound
summary
The talk explores encoding audio as images and implementing a browser-based music player that plays those pictures. It covers experiments with storing MP3-like audio data in JPEG-style visual formats and the techniques used to render and decode sound from images in the browser.
AI-written summary · about
chapters (40)
- 0:00Introduction
- 1:54Speaker background
- 2:38Visualizing sound
- 4:00Sound recording history
- 7:12Phonograph and vinyl
- 9:24Sound in cinema
- 10:49Soviet sound synthesizers
- 11:51Physics of sound
- 15:08Human hearing range
- 16:53How ear converts sound
- 17:14Nyquist theorem and digitization
- 20:00Digital sound recording
- 22:28Reading spectrograms
- 23:32Web Audio API
- 26:34Browser oscillogram
- 29:45Oscillogram demo
- 30:53Building spectrograms
- 32:18Frequency analyzer in Web Audio
- 33:01Fourier transform role
- 35:46Browser spectrogram
- 39:40Music track spectrogram
- 41:01Aphex Twin trick
- 42:33Oscillogram vs spectrogram
- 42:57Embed image in sound
- 44:23Generate sound from image
- 47:58Conference logo sound
- 49:24Friend’s photo as sound
- 50:31Inspiring projects
- 51:35Spectrogram applications
- 51:56Music note recognition
- 53:03Steganography in spectrograms
- 54:10Sound to image conversion
- 55:20Music to image
- 56:26Play image as sound
- 57:31Transformed image sound
- 58:13Extra ideas
- 58:36Triangle sounds
- 59:43Audio compression via images
- 1:00:26Technology summary
- 1:01:15Thanks and Q&A
