Patterns in using AI/ML to build a SaaS product(ARC302)
summary
Maps multi-tenant SaaS needs to concrete patterns: use vector DB + RAG for tenant context, store embeddings per-tenant, and inject a global tenant context to avoid wrong-domain results; add throttling and token-count checks in the control plane to prevent noisy-neighbor saturation; implement a model selector / routing that sends cheap models for simple queries and larger models for complex ones; build a mutation loop that evaluates outputs across multiple models and mutates prompts from aggregated thumbs-up/down feedback, which cuts fine-tuning costs and improves quality at scale.
AI-written summary · about
chapters (8)
- 0:00Introduction to AI/ML in SaaS
- 2:35Why combine SaaS and Generative AI
- 6:35Foundation models and customization
- 15:45Tenant-specific generative AI experience
- 22:10Beekeeper’s generative AI journey
- 31:05Evaluating and improving AI responses
- 37:03Optimizing costs with model routing
- 39:38Key takeaways and next steps
related talks
26mPatterns of ML Models in Production
38mReal Time ML Pipelines in Multi-Tenant Environments
31mSaaS Migration: Moving From Single-Tenant to Multi-Tenant on AWS
14mPerformance Optimization in High-Volume Transaction Systems
21mBuilding Scalable ML Systems: Strategies and Insights
49mImpact on business and society
50mDemocratizing AI - Back-fitting end-to-end machine learning
44mBuilding Multi-Tenant Compute Systems in the Enterprise