Confidential, self‑sovereign LLMs
summary
Argues that mobile phones and TEEs enable confidential, self‑sovereign LLM hosting instead of giant Manhattan‑scale data centers, claiming an existing network of 103,000 devices that already runs models up to 14B parameters. Describes sharding across upcycled phone mainboards (example cluster with 160 cores, 20 GPUs, 240 GB RAM), says TEEs prove real hardware and encrypt workloads, and proposes combining TEEs + open‑source models + Web3 auditability to make confidential decentralized LLMs practical.
AI-written summary · about
chapters (9)
- 0:00Confidential self-sovereign LLMs matter
- 1:45LLMs impact job sectors
- 4:56Big Tech’s AI power concentration
- 8:23From LLMs to decentralized AI agents
- 11:36Mobile devices for AI compute
- 14:03Hardware security for AI execution
- 16:43LLMs on decentralized mobile hardware
- 18:50Building with TEE and open source
- 19:41Key takeaways and call to action
related talks
25mvLLM: Easy, Fast, and Cheap LLM Serving for Everyone
34mTeaclave: Making Computation on Privacy-Sensitive Data Safe and Simple
20mSecure, private LLMs in your cloud with Anyscale Endpoints
42mSponsored Session: Confidential Computing - New Capabilities for New Workloads
34mDecentralized Machine Learning
11mWhen Less Is More: A Technical Overview of LLMs and the Strength of Smaller Models
35mLocal LLMs on low end hardware - a practical perspective
21mTowards Building a Responsible Data Economy