On Human-robot Interactions via Multimodal Intent Modeling and Reachability-based Safety Assurance
summary
Presents a decision stack that learns a conditional multimodal model of human driving with a recurrent conditional VAE, builds a library of ~4,000 robot action sequences, and samples ~100,000 human responses per planning cycle in about 0.2–0.3 s to score motions and replan at 3–4 Hz. Safety uses offline Hamilton–Jacobi backward reachability to compute backward reachable sets and enforces safety-preserving controls via a linearized MPC constraint instead of hard switching. Validated on a full-scale steer-by-wire vehicle (X1) with an RC adversary; code on GitHub.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 2:05Autonomous Driving
- 7:19Ontological Approach
- 15:11Testing
- 18:13Integrating Safety Assurance
- 21:27Collision Avoidance Controller
- 28:31Safety Preserving Controls
- 29:47PID Control Problem
- 33:30Selfdriving cars
- 35:52Experimental data
- 37:13Current work
- 37:38Leveraging techniques
- 39:05Data rigidity
- 40:57Logical reasoning
- 43:06Modelbased enforcer learning
- 47:41Conclusion
- 49:25Collaborators
- 49:49Audience Questions
- 54:24Comments
related talks
57mReal-Time Decision Making in Hybrid Systems
08mHuman–AI Collab for Reliable Automotive Control Systems
24mUse of AI to quantify the "safety of autonomous vehicles"
49mUsing Deep Learning for Perception in Autonomous Systems: A Perspective from Control Theory
21mFuture in Motion: Robotics, Self-Driving Cars, and Humanoids Shaping Tomorrow
33mHuman-Like Behavior in Autonomous Systems
14mTowards Unobtrusive Physical AI: Augmenting Everyday Objects with Intelligence and Robotic Moveme...
28mPreference Learning from Minimal Human Feedback for Interactive