From an AI Model in a Notebook to a Real-Time Service: Let’s Architect!
summary
An example-driven design walkthrough for turning a trained AI model in a notebook into a low-latency HTTP service. It covers differences between batch preprocessing used by data scientists and streaming/online preprocessing needed in production, latency-reduction techniques, data-ingestion architecture examples, and other practical steps to move from POC to production deployment.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:37Training vs inference
- 5:46Production goal
- 8:12Simple prediction API
- 11:43Load issues critique
- 12:45Message broker decoupling
- 14:42Latency vs throughput
- 20:18Data preprocessing
- 25:44Solutions summary
- 29:39Advanced perspectives
- 31:05Summary
- 32:52Q&A
related talks
20mFrom notebooks to production - bringing devops to scientists
26mEngineering Excellence: Platforms and Architectures for Real-Time AI Analytics
16mOperationalizing AI in Finance: Real-Time Analytics
41mBeyond the Algorithm: What Makes Machine Learning Work?
36mApplied AI on the edge
25mFrom Data Exploration to Modeling to Deployment
54mArtificial Intelligence seen from the software development lifecycle perspective
19mReal-Time AI: Low Latency Solutions for Interactive Applications