Sign Language Recognition Improvements Since the London Keynote
summary
Prototype moves from static snapshots to dynamic gesture tracking with 21 hand keypoints (X/Y/Z) fed through MediaPipe into ML models for fast handshape inference, and into an LLM for contextual disambiguation; inference runs via KServe inference service and Kubeflow, version 2 is running on Kubernetes. Caching to a key-value store cuts LLM token costs, but LLM latency (P95/P99) and occasional mis-decodings (misspelled names) require fallbacks and human interpreters.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:29Sign language recognition challenges
- 3:59LLMs with ML models
- 6:28Recognition platform evolution
- 9:18ASL models and Kubernetes
- 12:31AI trust and accessibility
- 13:56Industry improvements
- 16:00Real-time AI vs interpreters
- 17:25Key takeaways
- 18:26Acknowledgments
- 20:10Q&A
related talks
15mKeynote: Empowering Accessibility Through Kubernetes: The Future of Real-Time Sign Language Interpretation
24mSIGN TO TEXT Unleashing the Power of Google AI for real time ASL transcription
35mHow deep learning can help interpret American Sign Language
40mKeynote: Open-source Multimodal AI in the Wild
23mKeynote
32mEverything you wanted to know about VLMs but were afraid to ask
20mSGLang: An Efficient Open-Source Framework for Large-Scale LLM Serving
38mUnraveling Multimodality with Large Language Models