GAMING: Benchmarking Sustainable LLM Deployment on Edge Devices
summary
Framework benchmarks edge LLM inference across device, cloud CI/CD and edge engine layers, measuring p50/p95 latency, throughput, GPU/RAM utilization, power, energy and CO2. Testbed compares NVIDIA RTX 4060 and Jetson Orin Nano on an IoT environmental Kaggle dataset using quantized models; QN 2.5 and DeepSeek 1.5B show the lowest power and emissions, with QN 2.5 strong on Jetson and DeepSeek 1.5B lowest CPU/power on RTX. Claim: single LLM training emits ~300 tons CO2 and uses ~270 MW (reported).
AI-written summary · about
chapters (8)
- 0:00Sustainable LLM deployment intro
- 2:13Motivation for green AI
- 5:39GAMING framework overview
- 10:10Agent test bed results
- 13:20Energy and carbon findings
- 15:56Use cases comparison
- 16:38Next steps for research
- 18:24Collaboration opportunities
related talks
23mLLMs on the Edge: The Future of On-Device Intelligence
29mEnhancing AI with Edge Computing: A Leap Towards Affordable Performance
59mML and the IoT: Living on the Edge
28mProduction-Ready LLMs on Kubernetes: Patterns, Pitfalls, and Performa...
31mLow-Latency Solutions for Resource-Constrained Environments
19mThe Quest for the Greenest LLM
17mCLONE: Customizing LLMs for Efficient Latency-Aware Inference at the Edge
47mOwn Your LLM: Running Generative and Agentic AI Locally