Reward Augmentation in Reinforcement Learning for Testing Distributed Systems
summary
Paper proposes using reinforcement learning to bias randomized testing of real consensus implementations, implemented in Go, by adding a decaying exploration bonus that the update takes as the maximum with the Q-value and by supplying a sequence of predicates (waypoints) to guide exploration. Evaluation on RedisRaft, Etcd and RSL shows higher coverage and improved bug frequencies, with targeted waypoint runs improving coverage for 20 of 26 predicates and artifacts available for reproduction.
AI-written summary · about
chapters (16)
- 0:00RL for distributed systems testing
- 1:24Randomized testing challenges
- 2:28RL approach for testing
- 3:55Novel reward scheme
- 5:02Raft consensus overview
- 6:46State/action abstraction trade-offs
- 7:50Biasing toward interesting scenarios
- 11:00Experimental setup
- 12:03Results: state coverage
- 12:44Biasing toward subspaces
- 13:28Bug discovery frequency impact
- 14:24Q&A
- 16:11Error bars discussion
- 16:52Background and motivation
- 17:37Beyond distributed systems
- 18:20Summary
related talks
01h 02mThe Key Ingredients of Optimizing Test-Time Compute and What's Still Missing
33mReward Stress Tests for Agentic Post-Training with Ray
14mFeedback-guided Adaptive Testing of Distributed Systems Designs
56mA Tutorial on Reinforcement Learning II
15mTesting Consensus Implementations using Communication Closure
34mOffline Reinforcement Learning and Model-Based Optimization
52mReinforcement Learning in Recommender Systems: Some Challenges
12mDemystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical G...