Making Every Game and App More Accessible with .NET and AI
summary
A .NET Azure platform called Project Enabled Play turns voice, face expressions, gestures, phones and watches into keyboard/mouse/controller inputs across games and apps, shipping working prototypes: a SignalR-backed API, Arduino Leonardo HID for real USB input, Xamarin/Maui clients, Snapchat lens + QR bridge, Twitch bot and Xbox Adaptive Controller support. Speech moves from cloud Cog Services with ~1.5s latency to an offline predictive ASR 10 ms edge model (Raspberry Pi/ONNX), plus action queuing, macros and per-user auth/account linking for scale.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:10Accessibility in gaming
- 6:21Dark Souls with AI
- 8:50Voice commands for CoD
- 14:00Adaptive gaming scaling
- 21:45Enabled Play prototype
- 27:03AI optimization
- 37:33ONNX for cross-platform AI
- 42:50Face expression detection
- 53:12Real-world impact
- 54:58Q&A
related talks
42mMaking Every Game More Accessible with .NET and AI
57mApplied AI and Accessibility to Play Games in New Ways
36mMaking Technology More Accessible with .NET and AI
40mMicrosoft AI for Inclusion and Innovation
46mBuilding accessible experiences with AI
47mAccessibility in the era of generative AI
53mAI for the better
50mAccessibility Revolution! | Gaming's New Frontier