Acoustic-based Upper Facial Action Recognition for Smart Eyewear
summary
Glass-mounted prototype uses a customized Seeed 4-mic circular array plus earphones and a Raspberry Pi to emit inaudible OFDM and extract OFDM-based CSI estimation fused with PCA. Short-time Fourier transform yields 8-link time–frequency patterns fed to a 3-layer CNN with transfer learning personalization. Evaluation on 26 subjects with leave-one-subject-out training reports per-class recognition for six upper facial actions (cheek-raise, brow-raise, brow-lower, wink, blink, neutral) with average F1 ≈ 0.92.
AI-written summary · about
chapters (7)
- 0:00Start
- 0:18Motivation
- 2:19Implementation
- 3:14OFDM-Based CSI Estimation
- 3:59CSI Signal Processing
- 4:39UFA Recognition
- 5:31Evaluation
related talks
15mC-Face: Continuously reconstructing facial expressions by deep learning contours of the face ...
16mRVEdge-Vision: A Fully Open,Ultra-Efficient On-Device AI Platform for Smart Eyewear
30mTurning Glasses into Smart Assistants: Open Source Vision Language Models
23mPrediction of music pairwise preferences from facial expressions
20mLip-Interact: Improving Mobile Device Interaction with Silent Speech Commands
22mUbicoustics: Plug-and-Play Acoustic Activity Recognition
20mAdvancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography
11mAmbient Intelligence and Emotionally Intelligent Devices