MyoSign: enabling end-to-end sign language recognition with wearables
summary
A wearable system captures inertial and muscle signals from a Myo armband (three IMU axes plus 8 EMG channels) and feeds a multimodal CNN into a bidirectional LSTM with CTC for end-to-end ASL recognition. Evaluation on 15 volunteers uses 70 words and 100 sentences (≈10,000 word samples); user-independent accuracy is 93.7% word, 93.1% sentence, and 92.4% on unseen sentences, demonstrating non-intrusive continuous recognition without pre-segmentation.
AI-written summary · about
chapters (8)
- 0:00Introduction to MyoSign
- 2:17MyoSign system overview
- 4:06Key challenges in sign recognition
- 5:41Deep learning network design
- 8:16Dataset and user-independent tests
- 10:27Results and conclusion
- 11:08Q&A: English vs ASL output
- 13:39Q&A: Vocabulary size discussion
related talks
35mHow deep learning can help interpret American Sign Language
06mWrite, Attend and Spell: Streaming End-to-end Free-style Handwriting Recognition Using Smartwatches
24mSIGN TO TEXT Unleashing the Power of Google AI for real time ASL transcription
30mBuilding a Sign-to-Speech Prototype with TensorFlow, Pytorch, and DeepStack
08mIndian Sign Language Recognition (ISLAR)
06mWR-Hand: Wearable Armband can Track User's Hand
27mSign Language Recognition Improvements Since the London Keynote
26mBuilding a Sign-to-Speech prototype with TensorFlow, Pytorch and DeepStack: How...