Gemel: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge
summary
Gemel is a system that jointly retrains and merges layers and weights across edge vision models so one copy of shared layers fits in GPU memory, combined with a cloud planner that finds mergeable configurations and an altered edge scheduler that places models to minimize swaps. It evaluates on 15 workloads, claims up to 60.7% memory reduction and improves accuracy versus time/space sharing (uploader: 8–39%), processes many more frames by cutting swap delays using a heavy-hitter layers priority for merging. Source code is available.
AI-written summary · about
chapters (7)
- 0:00Introduction to video analytics
- 1:05Challenges of edge deployment
- 2:21GPU memory bottlenecks
- 4:25The case for model merging
- 6:06Shared layer architecture
- 8:47Geml design and algorithm
- 12:29Evaluation and conclusion
related talks
16mGemma 4: Compacting Intelligence for the Edge
17mEkya: Continuous Learning of Video Analytics Models on Edge Compute Servers
19mGemma 4 Deep Dive
01h 56mEdge AI Systems
24mAccelerating AI on Edge
20mScaling Edge AI with Elixir and Nerves
17mGemino: Practical and Robust Neural Compression for Video Conferencing
14mRegion-based Content Enhancement for Efficient Video Analytics at the Edge