Firefly: Untethered Multi-user VR for Commodity Mobile Devices
summary
System pre-renders panoramic 360° color+depth frames into tiled video (office map 137 GB, museum 99 GB), serves tiles with viewport-adaptive fetching plus linear-regression motion prediction (50 ms history → 150 ms ahead), and runs a server-side AQC every 15 ms to pick quality levels. Prototype is 17,400 LOC, replays 15-user traces on 5 phones + 10 Raspberry Pi, uses one AP/server under 200 Mbps, and sustains 15 users at 60 FPS 90% of the time with low stalls.
AI-written summary · about
chapters (21)
- 0:00Intro
- 0:33Motivation
- 2:31State-of-the-art
- 3:14Firefly
- 3:59Challenges
- 4:46Outline
- 4:56High Level Architecture
- 6:44Offline Rendering Engine
- 8:16How to fetch tiles?
- 9:33Understanding VR Motion Predictability
- 10:19VR User Motion Profile
- 12:12VR User Stationary Periods (SP)
- 13:58System Architecture
- 17:44Dynamic Foreground Objects
- 19:07Implementation and Evaluation Setup
- 20:07Overall Performance Comparison
- 21:55Micro Benchmarks
- 22:20Case Study - Adaptiveness to Number of Users
- 23:24Case Study - Adaptiveness to Available Bandwidth
- 24:06Energy Usage and Thermal Characteristics
- 24:26Summary
related talks
06mMobileportation: Nomadic Telepresence for Mobile Devices
28mAn Introduction to WebVR
07mBringing back human interaction to a digital world
33mCinematic Virtual Reality with 360 degree movies
38mMultiplayer WebVR
23mParallel Eyes: Exploring Human Capability and Behaviors with Paralleled First Person View Sharing
27mVolumetric Performance Capture and Playback: A Workflow with Python
38mDesigning Augmented Reality Systems for Speed