Building Web-Controlled AR Experiences at the NYTimes
summary
NYTimes builds a web-controlled AR system that renders natively to avoid frame drift and let editorial publish new experiences without app releases, targeting ARCore and ARKit and shipping an Android experience in mid-March. It exposes a web-controlled with native rendering interface, implements a command/result JSON loop to control a scene graph nodes tree, and pushes spontaneous updates via an low memory event listener. Tradeoffs include expensive webviews, heavy texture memory, and the need to version the native interface.
AI-written summary · about
chapters (42)
- 0:00Intro
- 0:19Ramona Harrison
- 0:42Introduction
- 1:45AR Core
- 2:30Availability
- 3:00Fundamental Concepts
- 3:36Motion Tracking
- 5:28Costumes
- 6:01Feature Points
- 6:42Limitations
- 7:31Light Estimation
- 8:01User Interaction
- 9:11Anchors
- 10:11Experience Building
- 10:38Our First Goal
- 11:14Our Second Goal
- 12:12Three General Approaches
- 12:36Native Approach
- 12:55AR Sample Project
- 13:32Native AR
- 14:24Web Rendering
- 16:10Advantages
- 18:04WebControlled Native Rendering
- 18:20WebControlled Native App
- 18:49Native App agnostic
- 19:16Benefits
- 20:33Scene Graph
- 22:13Communication Loop
- 22:28Commands
- 22:58OnCue
- 23:13Command Structure
- 23:38Required Data
- 24:49Web Interface
- 25:40Events
- 25:58Tracking State
- 26:17EventListener Loop
- 26:40Add EventListener
- 27:01OnEvent
- 27:16Event Messages
- 28:30Lessons Learned
- 30:13Method Tracing
- 30:42Conclusion
related talks
22mThe Case for Augmented Reality on the Web
26mExploring Augmented Reality on the Web
47mAugmented Reality on the Web
23mPerformant Augmented Reality for the Web
57mOn the path to WebAR
12mWeb Standards will make augmented reality awesome!
23mWebAR: the Augmented Reality for your Browser
24mGet into AR/VR with Single Implementation