LLMs Behind Bars: Sandboxes at Scale for AI on a Short Leash
summary
CodeRabbit runs NSJail sandboxes backed by OverlayFS and reports ~700 ms startup (NSJail adds 50–100 ms), about half a million sandboxes per day and peaks of 33,000 per hour, each lasting 3–10 minutes. Sandboxes run ~50 third‑party tools, commands average ~250 ms. Architectures compared: container+VM (recommended), microVMs like Firecracker VMs, gVisor, and Kata. Secrets are tokenized or proxied, egress is restricted, and lessons are concrete: reuse layers to cut cost, prefer VM-backed containers for isolation, and expect exfiltration attempts.
AI-written summary · about
chapters (12)
- 0:00Agentic sandboxes intro
- 3:29Why sandboxes for AI agents
- 7:43Two sandbox architectures
- 10:12Code Rabbit’s sandbox details
- 13:48Scaling 500K sandboxes/day
- 15:32Agent workflow in Slack
- 18:00Linux primitives sandbox
- 21:10MicroVMs vs containers
- 24:00Kubernetes sandboxing
- 26:32Protecting secrets in agents
- 29:21MCP vs shell scripts
- 30:02Sandboxing takeaways
related talks
25mPlease Sandbox your AI Agents
38mWhy, and how you need to sandbox AI-Generated Code?
22mSandboxes all the way down - A hitchhiker's guide to platform...
29mIsolation without Containers
24mContained Intelligence: Secure Sandboxing for Gen AI Code
47mSecure Sandboxing in a Post-Spectre World
12mDocker Sandboxes for AI
25mCloud-Native Sandboxes for Microservices: Understanding New Threats and Attacks