InSituTale: Enhancing Augmented Data Storytelling with Physical Objects
summary
Prototype InSituTale enables presenters to manipulate visualizations by physically moving objects, claiming real-time augmented data storytelling with tangible controls. The system combines a depth camera, object recognition, and Vision-LLM to detect object state and map motions (e.g., bringing objects together clusters bars, vertical stacking creates stacked bars, peeling a banana triggers chart transformation). Design is grounded in a survey and workshops with nine HCI/VIS researchers and a user study with 12 participants that reports intuitive interactions, high utility, and engaging live demos.
AI-written summary · about
chapters (9)
- 0:00Introduction to InSituTale
- 0:47Motivation and research gap
- 2:11Design process and workshop
- 4:20System implementation and setup
- 5:00Core interaction techniques
- 7:26Advanced interactions with vision models
- 8:30User study and findings
- 9:56Reflections and future work
- 11:00Q&A session
related talks
09mTangibleData: Interactive Data Visualization with Mid-Air Haptics
14mCombining Shape-Changing Interfaces and Spatial Augmented Reality Enables Extended Object Appearance
42mSpatial UX & Spatial AI [Keynote]
23mFuture of User Interaction: Mixed Reality, AI and IoT
23mExperiential Augmentation: Uncovering The Meaning of Qualitative Visualizations when Applied to ...
12mPhenomenological study of Augmented Reality
19mInvestigating the Use of Spatial Interaction for 3D Data Visualization on Mobile Devices
09mAdvanced Augmented Reality interactions