Building generative AI-powered products
summary
Lexer builds an embedded generative-AI feature called Lexi assistant that analyzes billions of customer events, using pre-aggregation and an early RAG pattern to avoid sending raw PII. Initial rollout suffers hallucinations, high token cost and low adoption (about 60% opt-in), then migrates to Amazon Bedrock running Claude, Nova and Llama 3 so inference stays in-region. Outcome: higher NPS and ease-of-use, 4.5 G2 rating, measurable time savings for marketers; ongoing work on tool-calling, proactive alerts and smaller-model cost optimizations.
AI-written summary · about
chapters (26)
- 0:00Introduction and setup
- 1:16Early adopters of generative AI
- 2:20Lexa’s customer data platform
- 4:07User challenges in data analysis
- 5:50First experiments with GenAI
- 7:35Building Lexi AI assistant
- 9:21Architectural and team challenges
- 10:44Privacy and security concerns
- 11:46Early RAG implementation
- 13:09Governance and user feedback
- 14:15Success metrics and customer adoption
- 14:36Transition to Amazon Bedrock
- 15:59Data privacy and security with Bedrock
- 16:20Future plans for Lexi
- 18:06Common generative AI challenges
- 18:28Security and privacy in Bedrock
- 20:55AI safety and guardrails
- 21:36Model selection and evaluation
- 23:23Prompt management and testing
- 24:25Optimizing model and prompt
- 25:49QA and user acceptance testing
- 26:30Cost optimization strategies
- 27:12Multi-model approach with Bedrock
- 28:59Inference and batch processing
- 29:20Prompt caching for efficiency
- 30:02Closing and survey request
related talks
18mGenerative AI in Financial Industry
48mUnlocking LLM Potential: From Exploration to Integration
41mAn Art, a Science, or Your Next Job Title?
36mGENAI-Powered chatbot to increase team efficiency
29mCustomizing genAI applications for your business
42mUsing LlamaIndex for building Generative AI apps
45m9 Lessons Learned from Deploying GenAI at Scale
41mGenerative AI: bliss or miss?