Interactive Room Capture on 3D-Aware Mobile Devices
summary
System uses a Google Tango 3D‑aware mobile device with depth camera and an augmented‑reality UI to tap and segment dominant object planes, send a photo to CNNs trained on ShapeNet (≈6,000 chairs, 8,000 tables), retrieve CAD models and align them by matching planes; full room capture finishes in ~5–8 minutes. A 10×2 within‑subject study shows faster and lower placement error than Homestyler. Works well for planar furniture but fails on plants/curved chairs and causes device‑holding fatigue; uses plane alignment and ShapeNet retrieval.
AI-written summary · about
chapters (15)
- 0:00Introduction and motivation
- 1:11Limitations of existing tools
- 2:353D-aware mobile devices
- 3:39Interactive capture workflow
- 5:04Related research context
- 6:08Key assumptions and simplifications
- 7:57Furniture surface capture demo
- 9:22Deep learning for model retrieval
- 11:04Placement algorithm details
- 12:28User study and evaluation
- 13:53Qualitative user feedback
- 14:35Generalization to other scenes
- 14:55Limitations and future work
- 15:58HoloLens proof of concept
- 17:03Conclusion and Q&A
related talks
41mComputer Vision for Remote AR
13mFlyMeThrough: Human-AI Collaborative 3D Indoor Mapping with Commodity Drones
15mGaussianNexus: Room-Scale Real-Time AR/VR Telepresence with Gaussian Splatting
17mData Challenges with 3D Computer Vision #bbuzz
15mMeCap: Whole-Body Digitization for Low-Cost VR/AR Headsets
35mDeep learning for 3d human motion capture
15mDepthLab: Real-time 3D Interaction with Depth Maps for Mobile Augmented Reality
33mSee More, Be More