Challenges in Reward Design for Reinforcement Learning-based Traffic Signal Control
summary
Claim: a direct emission signal is a poor reward for bounded DRL agents. Experiments use SUMO with the default emission model, a DQN agent backed by a tiny 2‑layer MLP (64 neurons), 5s phase steps, and heterogeneous Bernoulli traffic. CO2-based penalty fails to train DQN — emissions hover near ~140,000 g/h. A linear queue+brake penalty yields fast learning and cuts emissions to ~75–80,000 g/h, but performance is highly sensitive to the weighting of combined rewards.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:47The Importance of Aligning Powerful AI Systems
- 3:02Reinforcement Learning Example: Cliff Walking
- 5:05Aligning TSC Agents with Rewards
- 6:11Objective: Minimizing CO2 Emission at a Signalized Intersection
- 7:36Reinforcement Learning Setup
- 9:24Training the Neural Network - Deep Q-Network (DQN)
- 11:53Motivation - Uninformative Emission Penalty
- 13:05Informativeness and Expressiveness for Alignment
- 15:01Findings Comparing Rewards
- 17:51Findings - Rewards are sensitive to parameterization
- 18:33Conclusion - Informativeness and Expressiveness are necessary
- 19:52Technologies that helped a LOT
related talks
15mA Comparison of Reinforcement Learning Agents Applied to Traffic Signal Optimization
20mEnergy Optimized Green Light Assist in Varying Traffic Scenarios Using Reinforcement Learning
19mTestbed for Development of AI Solutions to Boost Operational Efficiency in Transportation Networks
11mUsing Deep Reinforcement Learning to Coordinate Multi-Modal Journey Planning with Limited ...
12mDemystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical G...
18mMaximizing Luck in Reinforcement Learning
56mA Tutorial on Reinforcement Learning II
29mReinforcement Learning with Human Feedback