Inside-Out Instrument Tracking for Surgical Navigation in Augmented Reality
summary
Inside-out 6DOF tracking of retro-reflective marker spheres using only the HoloLens 2 front stereo cameras, with stereo triangulation for initialization and a SCAAT-EKF (single-constraint-at-a-time extended Kalman filter) for real-time updates. The system achieves about 1.7 mm / 1.11° static and 1.9 mm / 1.2° dynamic error, runs at ~1.5 ms per frame, outperforms monoscopic ArUco tracking, works best at 20–60 cm, and replaces external optical trackers for navigation tasks that do not require sub-millimeter precision.
AI-written summary · about
chapters (9)
- 0:00Introduction to AR tracking
- 1:11Limitations of existing trackers
- 2:04Our marker sphere tracking approach
- 3:15Stereo vision and SCAAT filter
- 4:20Online tracking steps
- 5:57SCAAT filter mechanics
- 7:04Technical evaluation results
- 8:36Comparison with other methods
- 9:24Conclusion and future use
related talks
22mPinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality
53mA Thorough Look into Spatial Mapping with HoloLens
01h 05mChasing holograms - the future of HoloLens development
42mHow to Augment Your Reality
17mAugmenting Augmented Reality with Non-Line-of-Sight Perception
11mStabilizAR: Enabling Hands-Free Head Pointing while Mobile
44mBetween Two Realities: Testing AR / MR
09mSee a heart surgery being planned with HoloLens 2