Wrist2Finger: Sensing Fingertip Force for Force-Aware Hand Interaction with a Ring-Watch Wearable
summary
A wearable system combines a ring with 9-axis IMU and a watch with single-channel EMG plus a dual-branch transformer with cross-attention to jointly predict 3D hand pose and per-finger force in real time. Evaluation with 20 participants reports MPJPE 0.57 cm and fingertip force RMSE 0.213 N (r ≈ 0.76), ground truth from fingertip pressure sensors and MediaPipe, and ablations show removing IMU or EMG substantially degrades performance. Real-time Unity demo works.
AI-written summary · about
chapters (9)
- 0:00Introduction to Wrist2Finger
- 0:45Motivation for force-aware interaction
- 2:51System design and hardware
- 4:18Learning framework and model
- 5:19Data collection and evaluation
- 6:26Force estimation results
- 7:32Hand pose estimation results
- 8:17Comparison with baselines
- 9:22Conclusion and Q&A
related talks
07mAuraRing: Precise Electromagnetic Finger Tracking
17mAccurate and Low-Latency Sensing of Touch Contact on Any Surface with Finger-Worn IMU Sensor
17mEMPress: Practical Hand Gesture Classification with Wrist-Mounted EMG and Pressure Sensing
04mDualRing: Enabling Subtle and Expressive Hand Interaction with Dual IMU Rings
05mReady, Steady, Touch! — Sensing Physical Contact with a Finger-Mounted IMU
06mHow Subtle Can It Get? A Trimodal Study of Ring-sized Interfaces for One-Handed Drone Control
09mRotoWrist: Continuous Infrared Wrist Angle Tracking using a Wristband
06mIdentifying Contact Fingers on Touch Sensitive Surfaces by Ring-Based Vibratory Communication