Scaling Data Science Products, Not Data Science Teams
summary
Concur scales products into new countries without hiring more data scientists by shipping a single codebase with configurable features to segregated data centers, using modular microservices, online learning and synthetic warm-start datasets generated from existing traveler data to bootstrap models. Feature engineering pivots on language/character-set segmentation, PCA/eigenvector dimensionality reduction, and confidence thresholds to hide low-quality predictions. The deployed product ExpenseIt extracts vendor, amount, currency and date from receipts; fraud detection runs in an Audit service combining algorithms and human review.
AI-written summary · about
chapters (8)
- 0:00Start
- 5:43data science therapy: let's talk about your problems
- 7:36product design problem: managing user expectation
- 10:01software engineering problem: scale a code base
- 12:22mathy ML problem: feature design
- 14:01software engineering problem: test test test
- 15:16mathy ML problem: cold start
- 16:28product design problem: identifying feedback loops
related talks
41mData Science – The New Troublemaker in the Delivery Pipeline
38mScaling Data Science
31mScaling data science in the enterprise
41mThe success story of data science at Exact. Empowering the accountant of tomorrow.
29mBecoming a data-driven enterprise
39mProductionising Data Science at Scale
22mScaling analytics and AI: It's mostly about culture
27mEnterprise AI at Scale