Ray @ eBay: Pioneering a Next-Gen AI Platform
summary
eBay integrates Ray across batch inference, near‑real‑time feature pipelines, distributed training and real‑time serving, using Ray Data, Triton, fractional GPU and a standardized GPU SKU on Kubernetes with federation and custom GCS storage. Results: unified Python APIs cut handoff time by 50% development time, batch jobs reach ~96% GPU utilization and ~4x throughput, a risk pipeline goes from 25% to 75% GPU and 2.6x throughput, near‑real‑time sees 4x GPU and 7x throughput gains.
AI-written summary · about
chapters (12)
- 0:00Introduction to Ray at eBay
- 1:06eBay’s AI 2.0 mission
- 2:32AI platform architecture overview
- 3:37Model development and deployment flow
- 7:08Problems in the existing flow
- 10:06Ray integration for AI 2.0
- 10:28New AI 2.0 architecture with Ray
- 13:20Ray-powered notebook improvements
- 17:38Batch inference with Ray
- 22:16Near real-time inference pipeline
- 28:40Multi-data center Ray deployment
- 30:09Results and improvements summary
related talks
22mRay, a Unified Distributed Framework for the Modern AI Stack
31mNext-Generation Data Science Workflows Using Ray
35mOperating a Production-Scale Physical AI Data Platform with Ray
03mRay: A Distributed Execution Framework for Emerging AI Applications
28mScaling Multimodal Data and Reinforcement Learning with Ray
24mBuilding Complex Data Analytics Pipelines with Ray
28mScaling AI & Machine Learning Workloads With Ray on AWS, Kubernetes, & BERT
44mPanel: ML Platform on Ray