Building Handwriting Recognition System for the NY Times Crossword
summary
A prototype handwriting recognizer for the New York Times Crossword emerges from an internal Maker Week hackathon and runs entirely on Android with an on‑device TensorFlow Lite CNN (~200–300KB model) after comparing Google ML Kit and classical edge detectors. Training uses the EMNIST dataset plus synthetic augmentation (tens of thousands → millions of warped images) and hyperparameter tuning to reach ~91% validation accuracy; practical work includes a 2,000ms touch lock, a QA correction mode, and mitigations for model drift and mode collapse.
AI-written summary · about
chapters (14)
- 0:00Handwriting recognition project
- 3:58Project background
- 6:47Input mechanism
- 12:28Pre-processing images
- 16:00Testing challenges
- 17:46Edge detection attempts
- 22:01Google MLKit exploration
- 24:13Custom TensorFlow Lite model
- 25:56Digit recognition training
- 30:56Data augmentation
- 34:07Model visualization
- 35:30Model architecture
- 38:01Next steps
- 39:24Q&A
related talks
25mBuilding a Handwriting Recognition System for the New York Times Crossword
40mBuilding a Handwriting Recognition System
44mHello Deep Learning
06mWrite, Attend and Spell: Streaming End-to-end Free-style Handwriting Recognition Using Smartwatches
19mReacTex: using React Native and Neural Networks to recognize handwritten equations
45mPrototyping with Jetpack Compose
51mDeciphering doctor's handwriting with deep learning
01h 07mDeep Learning with TensorFlow