Alternative Smartphone Interactions based on Sensor Data
summary
A concrete claim: sensor data available to JavaScript can augment keyboard and touch input. The demo glues DeviceOrientation and DeviceMotion plus touch events and a Dollar 1 gesture recognizer to a real search index of ~1 billion items and adds voice via the Web Speech API (Chrome/Android uses Google service). Results are mixed: tilt reliably switches views and subtle scrolling works, gestures are noisy and hard to draw while scrolling, and speech recognition works but is error prone.
AI-written summary · about
chapters (7)
- 0:00Start
- 0:22Alternative Smartphone Interactions
- 4:37Demo Search Engine
- 8:21One Touch Gesture Recognizer for 2d Gestures
- 10:13Device Orientation Event
- 11:52Create a Higher Order Gesture
- 14:53Speech
related talks
13mEnhancing Mobile Experiences with Device APIs
34m2023's most interesting browser APIs
21mI Didn't Know The Browser Could Do That!
18mExploring alternative interactions in JavaScript
31mThe power of mobile web browsers
37mBrowser APIs: the unknown Super Heroes
19m"Bringing Voice First Experiences to the Web"
39mI didn’t know the Browser could do that