Project Bonsai: How Microsoft is Using Ray and RLlib Inside a Machine Teaching Service
summary
Project Bonsai uses RLlib and Ray for distributed training and scales many simulators to build deployable control models called "brains." A domain-specific program written in English/Inkling declares a concept graph, staged lessons, goals (e.g., keep a ball centered, avoid a red obstacle), and randomized contexts (ball radius, initial velocity). The AI crafts rewards from those goals, trains policies across many simulators, and exports brains as containers to run on hardware (Moab), with live metrics like goal satisfaction and policy test evolution.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 3:01Machine Teaching
- 3:55Autonomous System
- 4:49Workflow
- 5:29Referral Learning
- 6:47Enforcement Learning
- 7:30Flexibility
- 8:31Dynamic Programming
- 9:39Distributed Training
- 10:16Demo
- 12:11Teach
- 15:30Train
- 17:43Summary
related talks
36mSkyRL: Democratizing RL Training for Researchers
17mMatineh Shaker, Artificial Intelligence Scientist, Bonsai
25mIntroducing Amazon SageMaker Kubeflow Reinforcement Learning Pipelines for Robotics
19mTraining Agentic Reasoners
30mTransforming Grab’s AI Platform with Ray
29mRay:A Distributed Computing Platform for Reinforcement Learning
44mPanel: ML Platform on Ray
23mUsing RLLib in an Enterprise Scale Reinforcement Learning Solution