Caravan: Practical Online Learning of In-Network ML Models with Labeling Agents
Talkat 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24) · 2024-07-10..07-12·
summary
CARAVAN is a prototype system that uses labeling agents (off‑the‑shelf foundation models like GPT‑4 via API) to generate noisy labels (70–80% accuracy) for online retraining of tiny in‑network models running on a Tofino switch + FPGA + server. It introduces an accuracy proxy metric to trigger selective retraining, processes 35M packets at 500k pps in experiments, yields a 30.3% average F1 improvement over offline models, cuts GPU time 61.3% and backend CPU 50–56%, and successfully keeps models current.
AI-written summary · about
chapters (13)
- 0:00Caravan introduction and goals
- 2:09ML traffic analysis challenges
- 3:55Small in-network model limits
- 5:42Online learning with large models
- 6:25Large models as labelers
- 7:54GPT-4 for data labeling
- 8:37Accuracy proxy for retraining
- 10:44Caravan workflow and components
- 12:09Implementation and evaluation
- 13:11Intrusion detection experiment
- 15:17Selective retraining impact
- 16:00Caravan scope and limits
- 17:03Conclusion and resources
related talks
51mSpeeding up deployment of machine learning models
31mReliable Data for Large ML Models: Principles and Practices
46mArchitecting Efficient Agents with Semantic Routing and Mixture-of-Tools
57mMachine Learning in the Age of IoT
17mEffective AI Agents Need Data Flywheels, Not The Next Biggest LLM
41mAI applications in autonomous driving
04mManaging Machines
38mBuilding robust machine learning systems