Neural MMO: Building a Massively Multiagent Research Platform with Ray and RLlib
summary
Neural MMO is an open research environment built with Ray and RLlib that runs up to 1024 agents in procedurally generated persistent worlds and supports long horizons, progression, combat, and trading systems. It uses an object-based observation space with attention encoders, segmented trajectory batching for variable lifespans, and scalable action lists including target selection. Experiments train on a single RTX 3080 and 32 CPU cores, collecting over a billion observations and producing interpretable emergent strategies that converge reliably.
AI-written summary · about
chapters (9)
- 0:00Introduction to Neural MMO
- 1:48Challenges in multiagent research
- 3:55Designing observation and action spaces
- 7:01Transition to RLlib infrastructure
- 8:47Neural MMO as an MMO research platform
- 10:31Using Neural MMO for research
- 13:19Interpreting agent behaviors
- 16:07Achievement systems and future plans
- 17:50Upcoming competition and resources
related talks
21mAdvanced Ray RLlib: New Features and New Possibilities
29mMiles: Enterprise-Facing Agentic RL Framework
28mScaling Multimodal Data and Reinforcement Learning with Ray
27mLessons from Unifying Simulation and RL Training
01h 09mMinecraft as AI Playground and Laboratory
29mRay:A Distributed Computing Platform for Reinforcement Learning
20mProject Bonsai: How Microsoft is Using Ray and RLlib Inside a Machine Teaching Service
24mRay: A System for High-performance, Distributed ML Applications