Digitizing 125 Years of Images
summary
Project digitizes a basement “morgue” of 6–8 million photographs and 850,000 index cards starting in 2018, scanning fronts and backs and piping images through Cloud Storage and Pub/Sub into Kubernetes services backed by Postgres with Golang workers and Imagemagick. OCR and handwriting models from multiple providers increase recall by tens to hundreds of percent, a custom card‑to‑folder parser maps indirect references into searchable tags, the internal search works for newsroom use but remains closed for rights reasons.
AI-written summary · about
chapters (32)
- 0:00Introduction
- 3:40Importance of the Archive
- 5:13Index Cards
- 7:38Step Zero
- 8:08Step Fifty
- 8:31Metadata
- 9:25Object Detection
- 13:11Facial Recognition
- 13:27John Lennon Yoko Ono
- 13:45Face Recognition
- 15:52Amelia Earhart
- 17:22Folder Covers
- 18:01Card Catalog
- 19:48Cart Defects
- 20:03Overlap
- 20:20Crooked Corners
- 20:40Endless Quest
- 21:48Crooked Card
- 21:58Corner Detection
- 22:24Angle of Rotation
- 23:14White Spaces
- 23:56Another Example
- 24:11Text Recognition Bakeoff
- 24:39Text Recognition Test
- 25:17Handwriting Test
- 26:39Structure
- 28:33Pedigrees
- 29:25Example
- 30:20Other Improvements
- 30:46Visual Storytelling
- 31:15New York City Pride
- 31:49Hiring
related talks
25mScanning is the new spinning - Digitization & Open Knowledge initiatives around you
57mApertureDB: Designing a Purpose-built System for Visual Data and Data Science
39mOpening a treasure trove: The Historical Aerial Photography project
58mArt and Science With Six Million Timelapse Images and Python
52mDesigning data with a ‘post-infographic’ mindset
30mIn the service of the history. AI in archivistics.
23mWeb 3, Authenticity and the Post-Photographic Age
33mWrangling Internet-scale Image Datasets