SceneCtrl: Mixed Reality Enhancement via Efficient Scene Editing
summary
SceneCtrl reconstructs Hololens meshes, segments patches on-device, and lets users sketch-based selection to delete, copy, or move real objects; deleted areas are covered by reconstructed geometry textured from user photos using weighted linear blending and dynamic view-dependent textures generated in real time. Prototype runs interactively with 10 participants who learn the UI in ~15 minutes and report surprise that physical objects can be removed. Limitations: texture quality depends on captures and bright objects are hard to hide.
AI-written summary · about
chapters (12)
- 0:00Introduction to SceneCtrl
- 1:34Challenges in traditional scene editing
- 3:05Scene understanding for object selection
- 4:54Sketching for object selection
- 5:16Texture reconstruction for deletion
- 7:09Dynamic texture generation
- 7:33Live demo of SceneCtrl
- 9:25Redesigning rooms with virtual objects
- 10:11Comparison with traditional AR
- 10:34User study and evaluation
- 12:03Limitations and future work
- 13:33Q&A session
related talks
10mMixed Reality check
46mHoloLens 301: Advanced Mixed Reality Development and Best Practices
46mBuilding a Collaboration Service for the Hololens
17mDeveloping for the Hololens
40mHolographic Telepresence with Hololens
01h 08mDeveloping for Mixed Reality with HoloLens
22mBlood, Sweat and Hololens
16mSnapToReality: Aligning Augmented Reality to the Real World