Merge Realities, Multiplay Wonder: Expert Guidance on Mixed Reality Development
summary
Meta releases the Passthrough Camera API to public experimental last Friday, exposing Quest 3 forward cameras via Android Camera2 so apps can feed frames into ML models (example: YOLO compiled into Unity for object bounding boxes). Starship Home demonstrates durable surface-aware object landing and object permanence—playtests show most users remember object locations and even move furniture to accommodate virtual items. Spatial Ops ships after two years of development with anchoring and occlusion; Hello Dot and Paradot prototype texture sampling, wayfinding, and UGC workflows.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 4:13Breaking down mixed reality into Passthrough and Occlusions, Anchors and Scene
- 6:47Understanding mixed reality experiences
- 8:45Tips from Doug North Cook
- 17:55Passthrough Camera API intro
- 19:37Using Passthrough Camera API with Tommy Palm
- 25:20Exploring more Passthrough Camera API use cases with Alicia Berry
related talks
39mDesigning for the Living Room: The Spatial Architecture of Mixed Reality
10mMixed Reality check
36mMixed Reality - 3 Axes of Improvement
33mMixed Reality for Social Good
43mFrom Flatland to Spaceland
03mVirtual Real Spaces: In-Home Gaming Experiences Bridging Reality and Virtuality
38mFrom Flatland to Spaceland
21mWhy We Care About Mixed Reality From a Movie Studio Perspective