Building GenAI products: a product manager's perspective
summary
Built an agentic product that runs 60–90 minute technical pairing interviews, sold to enterprises and run in production with about 2,000 interviews so far, claiming one customer cut hiring cost by ~60%. System uses OpenAI off‑the‑shelf models (no fine‑tuning), keeps the same model for months, requires a human in the loop for every session today, exposes full transcripts and audit logs for traceability, has ~1,000 external users and a 4.3 CSAT. Testing remains largely manual.
AI-written summary · about
chapters (17)
- 0:00Building GenAI products intro
- 5:29AI agents explained
- 8:38GenAI in recruitment journey
- 13:46AI interviewer for tech hiring
- 18:08Building with vs without customers
- 20:14Key thing when building GenAI
- 23:01Do customers want GenAI?
- 27:00Fitting AI into workflows
- 30:08Transparency in GenAI products
- 33:59Human-in-the-loop persona
- 36:10Non-functional GenAI requirements
- 37:34UX in AI products
- 38:35Prompt engineering skill
- 41:48Testing GenAI challenges
- 43:29Prompt refactoring & automation
- 44:33Tool selection & simplicity
- 45:14Enterprise reliability focus
related talks
31mGenAI after the Hype: Transforming Organizations with GenAI-based Agents
01h 06mAI in the Trenches: Engineering Real-World Systems
23mScaling GenAI Applications and Agentic AI for Business Competitive Differentiation
17mGenerative AI: Keeping it real
37mGenAI Apps: Our Journey from Ideas to Production Excellence
25mBeyond the Hype: Real-World AI Strategies
56mPANEL: Unleashing the Future: The Impact of Gen AI on Development Productivity
16mBuilding Applications with AI Agents