SoundCraft: Enabling Spatial Interactions on Smartwatches using Hand Generated Acoustics
summary
A 1.5 × 1.5 inch smartwatch prototype with a microphone array and IMU localizes hand-generated sounds angularly and classifies gestures using a generalized MUSIC pipeline plus a GMM classifier. Localization reaches about 90% localization (azimuth >22.5°) and classification hits ~90% for 11 signatures, with 89%/88% in noisy settings. Demonstrated apps include one-handed scrolling, 2D pie menus, sketching and a two-player ping‑pong; limitations are sensitivity to transient noises, no distance estimation, and user variability for some gestures.
AI-written summary · about