Ben Lau, Quantitative Researcher, Hobbyist, NYC 2017
summary
Builds a self-driving car in the open-source TORCS simulator using a deep reinforcement approach (policy-gradient / actor-critic style) that inputs rangefinders, speed and track position and outputs steering, throttle and brake. Trains for a few hours on one track, validates on unseen tracks, and reports long runs (mentions ~200 km) but persistent oscillation (“waggle”) on some corners. Solves poor braking by reward shaping and forcing occasional collisions during training; source code is available (search PD PG TORCS).
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:14What is Reinforcement Learning? 3 classes of learning
- 1:02RL:Agent and Environment
- 3:53Policies
- 5:37Value Function A value function is a prediction of future reward
- 6:31Understanding Q Function
- 7:18How to find Q-function
- 8:43Deep Q-Network (DQN)
- 10:03DON Demo
- 11:12Approach to Reinforcement Learning
- 11:33Deep Policy Network Review. A policy is the agent's behavior
- 15:10The power of Policy Gradient
- 15:49Intuition:Value vs Policy RL
- 17:53Deep Learning Recipe
- 18:40Design of the reward function
- 20:03Training Set:Aalborg Track
- 20:47Validation Set:Alpine Tracks
- 22:08Learning how to brake
- 24:25Final Demo - Car does not stay center of track
- 26:35Future Application
related talks
52mBuilding a level 5 autonomous vehicle (in TrackMania)
23mRacing Robocars
22mDeveloping AI in Racing Games Using Reinforcement Learning
49mBuilding a self driving RC car
15mFrom the Racetrack to the Road - How Open Source Boosts Autonomous Vehicle Development
01h 51mLearn to Drive (and Race!) Autonomous Vehicles
27mPreparing for the Autonomous Vehicle Era
35mDingoCar: Perception module Small scale, self-driving car