Automating Full-Screen TalkBack for a Government App with Designer Collaboration and AI Agents — a Practical Experience
summary
An automated pipeline converts Figma designs into Android Jetpack Compose accessibility (semantics) code using Figma MCP, a custom Claude Code skill, and a TalkBack validation app. It extracts Figma annotations into Markdown mappings, generates component- and screen-level Compose semantics API code, and runs device tests that caught real TalkBack bugs (e.g., double reading when contentDescription and inner Text both speak, and liveRegion plus focusable causing double reads with external keyboards). The project was applied to 35 design-system components and ~30 screens and includes iterative updates to the rules/skill plus a component-only verification app.
AI-written summary · about
chapters (10)
- 0:00TalkBack automation project
- 3:08TalkBack in government apps
- 7:51Shared automation components
- 12:12Four-phase TalkBack workflow
- 18:20Figma instructions generation
- 21:52AI agents for TalkBack rules
- 27:36Testing TalkBack on devices
- 30:26TalkBack bugs and fixes
- 32:15Key takeaways
- 33:41Summary and next steps
related talks
19mCreating Narratives: Shaping TalkBack with Compose Semantics
43mBuilding Accessible Apps with Jetpack Compose
44mAccessibility with Compose : Experimentation & Live coding
34mDeep Dive into Compose Semantics
34mMulti-coding agent workflows
48mAccessibility is not rocket science!
34mFrom Figma to Quality Code: Code Connect + AI
17mNew options for accessible design