Robot Hand Driven by Computer Vision and Reinforcement Learning
summary
Project builds an apple‑picking pipeline using Gazebo and ROS for simulation, compares detection models (fails with YOLO pre‑trained on COCO, succeeds with a CNN trained on a custom dataset), and tests three RL approaches — a Tesla‑style two‑network attempt fails, an actor‑critic with dual CNNs partly succeeds, and a multi‑goal RL using direct future prediction works best in sim. Training runs ~440,000 iterations (~two weeks) and real tests use a DIY pipe arm, lasers for ranging, and papier-mâché apples because real fruit breaks balance; real‑world control is slow and brittle.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 0:45Project Apple Picker
- 1:59Reinforcement Learning
- 3:37Methods
- 5:51Practical Scenes
- 6:34Recognition System
- 8:57Real Apple
- 9:35First Prototype
- 10:05Robot Hand
- 10:29Crash
- 10:54Problems
related talks
21mVisualizing robot minds, and other fun stuff
16mKeynote: Robot Foundation Models
17mBeyond Automation: AI and the Evolution of Robotic Intelligence
18mRobotics: why now?
01h 02mThe Modern Trolley Problem - Responsible AI Principals
15mLeRobot – Lowering the entry barrier to AI for robotics
49mBuilding a self driving RC car
49mUsing Deep Learning for Perception in Autonomous Systems: A Perspective from Control Theory