Deep learning for 3d human motion capture
summary
Pipeline takes a single phone RGB frame and produces an animatable SMPL mesh by a CNN that predicts 23 joints heatmaps plus shape and pose vectors, feeds them into a joint regressor and SMPL decoder, and refines with an optimization using reprojection loss + Adam; shape space is compressed with PCA and linear blend skinning plus blendshapes handles soft‑tissue. Inputs are normalized to 224×224, target meshes use roughly 6870 vertices. Demo renders exportable meshes and runs fast enough for real‑time use.
AI-written summary · about
chapters (8)
- 0:00Introduction and background
- 5:43Simplify model overview
- 10:26Convolutional networks in Simplify
- 15:48Joint detection with Simplify
- 19:19SMPL model and 3D output
- 23:18Joint positioning and blend shapes
- 28:19Demo and practical use
- 31:51Q&A session
related talks
20mMoSculp: Interactive Visualization of Shape and Time
14mBack-Hand-Pose: 3D Hand Pose Estimation for a Wrist-worn Camera via Dorsum Deformation Network
41mComputer Vision for Remote AR
27mAnalysis of sports scenes using computer vision models
28mBuilding Motion Controlled Websites in JavaScript
07mFingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by ...
13mFlyMeThrough: Human-AI Collaborative 3D Indoor Mapping with Commodity Drones
05mTouchPose: Hand Pose Prediction, Depth Estimation, and Touch Classification from Capacitive Images