Scaling AI: Fault-Tolerant GenAI Solutions for Millions
summary
Elementor builds the AI site planner that serves ~1 million daily AI requests and exchanges ~10 billion tokens monthly, using OpenAI plus a multiorg proxy to multiply rate limits and a one‑click fallback to Anthropic Claude. Infrastructure includes a Redis-backed circuit breaker that auto-evicts failing OpenAI orgs and sends Slack alerts, plus an evaluation pipeline that logs every prompt/result and measures insert rate to find failures (e.g., transparent-background images) and auto-suggest fixes.
AI-written summary · about
chapters (8)
- 0:00Introduction and Overview
- 0:31Launching the AI Site Planner
- 1:23Challenges with OpenAI and Microsoft Azure
- 3:05Elementor's AI Journey and Achievements
- 5:32Handling 1 Million Daily AI Requests
- 13:33Building a Resilient AI Solution
- 23:53Continuous Improvement in AI Solutions
- 31:01Conclusion and Takeaways
related talks
28mScaling AI Infrastructure
22mAI
20mOperating Frontier Intelligence at Scale
13mScaling Generative AI Inference at Trillion-Token Scale
14mKeynote: Building the Infrastructure that Powers the Future of AI
17mAI in Production: Lessons Learned
15mScaling AI-Driven Financial Services: Ethical ML at Scale
36mHow to Get AI to Stop Being Weird and Actually Be Useful