Turn your smartphone into Nintendo Wii-like controller
summary
A web app reads the phone accelerometer via the device motion event, segments gestures by a threshold chosen from user tests (about 16–17 m/s²), and classifies motion sequences with K‑NN + Dynamic Time Warping as the distance metric. The demo maps left/right/up/down gestures to on‑screen arrows in real time, implemented in JavaScript, and the pipeline (segmentation → DTW distances → K‑NN) proves robust enough to control a simple game.
AI-written summary · about
chapters (10)
- 0:00Start
- 10:41Parametric vs non-parametric models
- 13:17Distance function
- 17:04Segmentation
- 18:14Finding activation threshold
- 20:45Dynamic Time Warping - pseudo-code
- 21:18DTW distances between gestures samples
- 22:39Web API for acceleration
- 24:24Handling data points
- 32:46Unsupervised learning recap
related talks
28mStrike a Pose - Gesture recognition in JavaScript w/ ML & Arduino
25mAlternative Smartphone Interactions based on Sensor Data
06mTowards Position-Independent Sensing for Gesture Recognition with Wi-Fi
22mStrike a Pose: Gesture Recognition in JS with Machine Learning
16mUnderstanding Grip Shifts: How Form Factors Impact Hand Movements on Mobile Phones
18mInfiniTouch: Finger-Aware Interaction on Fully Touch Sensitive Smartphones
19mHand gesture-controlled robot through the Internet
29mNatural User Interface using JavaScript