MeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets
summary
MeCap turns a smartphone's rear camera plus two hemispherical mirrors into a wearable whole-body capture rig that estimates 3D skeleton, hand skeleton, mouth state, apparel and an environment map. The prototype is a <$5 optical attachment that unwarps dual fisheye views, uses correspondence for depth, and runs pose and CNN classifiers; evaluation reports a 2.3 cm mean 3D Euclidean error, 96% gesture accuracy and end-to-end sensing for avatars and lighting.
AI-written summary · about
chapters (6)
- 0:00Introduction to MeCap
- 2:55How MeCap works
- 5:07Applications of MeCap
- 6:55Evaluating MeCap's accuracy
- 10:27Limitations and future work
- 11:13Q&A session
related talks
41mComputer Vision for Remote AR
06mMobileportation: Nomadic Telepresence for Mobile Devices
16mGetting hands on with VR
06mCoolMoves: User Motion Accentuation in Virtual Reality
05mTouchPose: Hand Pose Prediction, Depth Estimation, and Touch Classification from Capacitive Images
10mVirtual Reality platform for functional Magnetic Resonance Imaging in Ecologically valid conditions
14mOddEyeCam: A Sensing Technique for Body-Centric Peephole Interaction using WFoV RGB and NFoV ...
15mC-Face: Continuously reconstructing facial expressions by deep learning contours of the face ...