SIGN TO TEXT Unleashing the Power of Google AI for real time ASL transcription
summary
Uses Google tooling and Python to convert hand landmarks into letters by extracting MediaPipe hands landmarks with OpenCV, training an SVM classifier on image frames, and saving a TensorFlow-compatible model for inference. Trains on a Kaggle/Google-provided image dataset so motion letters J and Z are missing, evaluation reports precision/recall and F1 for each class, and real-time landmark tracking works but live letter output is unreliable in the demo due to dataset and iteration limits. Includes a runnable notebook and an exportable model meant for embedding in web or camera apps.
AI-written summary · about
chapters (12)
- 0:00Intro
- 1:30The Challenge
- 2:02Why talk about American Sign Language
- 3:50The importance of American Sign Language
- 5:18What is MediaPipe
- 5:50How does it work
- 6:36Endpoints
- 7:40Transcription Model
- 8:35Hands
- 13:34Evaluation
- 16:08Retraining
- 17:53Demo
related talks
35mHow deep learning can help interpret American Sign Language
30mBuilding a Sign-to-Speech Prototype with TensorFlow, Pytorch, and DeepStack
26mBuilding a Sign-to-Speech prototype with TensorFlow, Pytorch and DeepStack: How...
09mBeyond Captions A Practical Pipeline for Live AI-Generated Sign Language Avatars
27mSign Language Recognition Improvements Since the London Keynote
46mArtificial intelligence to the rescue of accessibility
27mIndian Sign Language Recognition(ISLAR)
47mBuilding smarter apps with Machine Learning, from magic to reality