Developing AI in Racing Games Using Reinforcement Learning
summary
Milestone builds racing AI using actor-critic neural nets trained for thousands of episodes with a catch-up reward (time gap to others) and quantized action outputs sampled during training but averaged in-game for smooth control. Observations are track borders, curvature samples, raycasts and vehicle physics. Training runs in a custom minimal game loop (no collision response), computational graphs are authored in C++/DSL, converted to C and compiled into a DLL for fast weight updates, and telemetry (Sauron eye) verifies behavior.
AI-written summary · about
chapters (15)
- 0:00Reinforcement learning intro
- 0:48Core RL agent concepts
- 3:45Exploration vs exploitation
- 6:39Catch-up reward system
- 9:04Actor-critic algorithm
- 10:10When RL fails
- 11:39Credit assignment challenge
- 13:04Short vs long rewards
- 14:49Reward design pitfalls
- 15:53Episode termination
- 17:23RL framework architecture
- 18:53Training environment tools
- 20:19Designer role in analysis
- 21:52Unexpected agent behaviors
- 22:13Final results & recommendations
related talks
52mBuilding a level 5 autonomous vehicle (in TrackMania)
29mThe Present & Future of AI and Games
24mDesigning a Video Game AI with Reinforcement Learning
02mDesigning a Video Game AI
48mMachine Takeover - Improving Your Game With Machine Learning & AI
15mReinforcement Learning and the Next Gen of AI
48mMachine Learning for Boardgame AI in Race for the Galaxy
37mAWS DeepRacer: Train, Evaluate, and Tune