Integrating High Performance Feature Stores with KServe Model Serving
Talkat Open AI + Data Forum @ Open Source Summit North America 2022 · on Thursday, 2022-06-23· at 14:05· Room 303/304 (Level 3), JW Marriott Austin, 110 E 2nd Street Austin, TX 78701·
summary
Integration shows Feast online store on Redis wired into KServe Model Mesh via a transformer that fetches live features with the go-based SDK and sends protobuf/gRPC requests to model runtimes. Deployment runs on a three-node Kubernetes cluster, serves multi-region driver-ranking models (San Francisco, San Jose), and the model mesh packs up to 20k models in two pods in tests while keeping single-digit millisecond latency under moderate load.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:34Agenda
- 0:58Background
- 3:43Feast
- 4:32Three Main Concepts
- 7:36Problems with KServe
- 10:34Deployment Scenario
- 13:42KServe Overview
- 16:38Model Mesh Architecture
- 18:11Scalability Test
- 19:14Highlights
- 20:34Making a Prediction
- 22:39Demo
- 23:36Cluster setup
- 28:02Recap
related talks
36mIntegrating Feast Online Feature Store with KFServing
40mServing Machine Learning Models at Scale Using KServing
36mWhat's New, ModelMesh? Model Serving at Scale
31mServing Machine Learning Models at Scale Using KServe
32mEnhancing the Performance Testing Process for gRPC Model Inferencing at Scale
28mExploring ML Model Serving with KServe (with fun drawings)
35mServing Large Language Models using KServe
31mFeast as a Cloud Native Data System: Bridging Batch Processing and Online Serving...