The Android Lens: Applying Mobile Forensics to AI Performance
summary
Loads Llama 3.2 (1B) onto a Raspberry Pi 4 and profiles Ollama from Go/C++ down into ggml and quants.c, using Perfetto, btop, LeakCanary-style checks and a Kotlin multiplatform harness. Patching mutexes, replacing naked locks with lock guards and fixing unbuffered channel sends yields higher token generation rates, overall CPU up but per-core temperatures down, memory drops under adversarial math, and concurrency stress tests cut runtime roughly in half while RAG hallucination scores mostly remain at 1. Llama 3.2 (1B) on Raspberry Pi 4 Ollama, ggml and quants.c fixes
AI-written summary · about
chapters (11)
- 0:00Sustainable AI and Android
- 3:11Frugal computing for AI
- 8:11AI performance bottlenecks
- 12:03Llama forensics monitoring
- 14:33Bottlenecks with pattern matching
- 17:01Deadlock exceptions in AI
- 22:39Blocking on channels in AI
- 25:50Optimizing matrix math loops
- 27:15Validating AI fixes with profiling
- 30:04Benchmarking AI improvements
- 34:14Final thoughts
related talks
35mOn‑Device LLMs on Android Devices 2025
03mFinetune Llama 4 Faster
27mAI Demystified Unveiling the Intelligence Behind the Machine
02h 52mLow Level Technicals of LLMs
27mFaster Than Fast: Networking and Communication Optimizations for Llama 3
35mLocal LLMs on low end hardware - a practical perspective
24mAccelerating AI on Edge
01h 35mSave your API Keys for someone else