Moxin: The All-In-One Tool to Run LLMs on Your Personal Device
summary
Moxin is an open source desktop app that runs LLMs locally with a Rust-based GUI and a backend using the WasmEdge runtime. Demo shows it running an 8B Llama 3 and a 1.8B Q model offline on a 16 GB M1 laptop, streaming responses and generating runnable Rust code, and claims a Mac Studio with ~200 GB RAM can host ~70B models. Model cards and benchmarks live on GitHub, downloads come from Hugging Face or local mirrors, and GPU acceleration uses Metal/CUDA where available.
AI-written summary · about
chapters (7)
- 0:00Moxin introduction
- 3:32Live demo on MacBook
- 13:00Why Moxin stands out
- 20:42Rust UI framework
- 24:59WebAssembly for AI
- 31:01Future plans
- 34:07Q&A
related talks
57mMoly - A pure Rust Explorer for Open Source LLMs
55mFast And Portable LLM Inference With WebAssembly And Rust
47mOwn Your LLM: Running Generative and Agentic AI Locally
35mLocal LLMs on low end hardware - a practical perspective
16mReact + WebLLM: Building Intelligent Applications with In-Browser Language Models
34mPrivacy First: Building LLM-Powered Web Apps with client side WASM
40mOpen Source LLMs in the Cloud: Scalable Solutions
30mWrite One Run Anywhere, But for GPUs