Dynamic Object Gaze Tracking with OpenCV
summary
Pipeline uses OpenCV SIFT+FLANN+RANSAC to find the static artwork inside each eye‑camera frame, computes a homography and applies perspective transform so gaze from eye‑tracking glasses maps to image coordinates, and produces kernel‑density heatmaps and a 3D pose trace via solvePnP camera pose estimation. Code uses FFmpeg and Docker for reproducible processing. Applied to Duke’s Nasher Museum artworks (including a 12‑image Family Tree set) and yields robust mappings except when frames are motion‑blurred, which simple smoothing corrects.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 4:42Problem statement
- 5:22Hardware and software
- 7:51Experimental setup
- 9:35Gaze mapping process
- 11:25Feature detection
- 14:21Affine transformations
- 18:16Gaze data on images
- 21:07Gaze data on video
- 23:423D gaze mapping
- 26:09Camera calibration
- 28:35Camera position solving
- 31:05Movement reconstruction
- 32:30Summary
related talks
19mAn Eye For Design: Gaze Visualizations for Remote Collaborative Work
38mWhere are we looking? Predicting human gaze using deep networks.
21mImmersive Trip Reports
21mVisualizing robot minds, and other fun stuff
20mMoSculp: Interactive Visualization of Shape and Time
35mLarge-Scale Video Intelligence
37mUsing Exploratory Data Analysis to Discover Patterns
30mIn the service of the history. AI in archivistics.