Into the Deep – or what would the 80s have done with depth sensing technology?
summary
Live iOS depth capture uses AVFoundation to produce AVDepthData backed by CVPixelBuffer of 16-bit half-float disparity values (disparity = inverse meters), so code converts half→float or reads single pixels for performance. TrueDepth (front) gives VGA-ish depth, back dual camera yields ~1/4 resolution and worse streaming quality. ARKit exposes ARFaceAnchor geometry plus 50+ blendShapes and eye‑gaze points. Use AVCaptureSynchronizer, Core Image for normalization and portrait blur, and throttle captures to avoid thermal throttling.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:46Live depth data capture
- 7:44Depth camera comparison
- 12:03Synchronizing depth and video
- 17:26ARKit face tracking
- 25:28Reading single depth values
- 29:10Normalizing depth data
- 32:41Core Image for depth
- 36:10Depth-based effects
- 40:03Portrait mode simulation
- 40:46Resources and closing
related talks
37mMobiconf 2018: Tobias Due Munk, Developmunk
15mDepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality
15mDepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality
29miPhone Facial Capture with Unreal Engine
23mMultimodal Video Face Liveness: A Better Alternative?
41mComputer Vision for Remote AR
35mAugmenting Faces and Images
33mSee More, Be More