Visualizing robot minds, and other fun stuff
summary
Convolutional neural networks trained on a 3-GoPro dataset classify trail position and actually drive a drone as a proof of concept, later squeezed to run 100 fps on-board inference on 25 g nano drones. Animations compare three models against human ground truth, show two-drone observer/target tracking, bracelet pointing vs joystick trajectories, a lidar point‑cloud person detector using temporal history, and a full-size robot handing out chocolates; code uses Python, matplotlib, ROS, pio and pyunk.
AI-written summary · about
chapters (15)
- 0:00Robotics research introduction
- 1:00Drone trail-following experiments
- 3:27Neural network outputs for drones
- 4:29Person-following drone
- 5:55Nano drones at 100 FPS
- 7:40Visualizations reveal robot behavior
- 8:44Drone control with bracelet
- 10:533D drone control experiments
- 11:58Robot serving chocolates
- 12:42Full-size robot interactions
- 14:09Lidar human perception
- 15:14Tesla self-driving insights
- 16:37Inverse kinematics animations
- 18:06Ball pit physics simulations
- 19:48Key takeaways and tools
related talks
45m(Deep) Learning to Fly
18mRobotics: why now?
48m(Deep) Learning to Fly
15mRobot Hand Driven by Computer Vision and Reinforcement Learning
52mFlying services with a drone
04mAI and the Robotics Revolution
49mBuilding a self driving RC car
49mUsing Deep Learning for Perception in Autonomous Systems: A Perspective from Control Theory