Moly - A pure Rust Explorer for Open Source LLMs
summary
A desktop app that runs open‑source LLMs locally, built entirely in Rust + Wasm with a swap‑in .wasm model runtime and a Rust backend that registers three callbacks for I/O. Components include a Rust-native chatbot UI, a LlamaEdge inference engine for token streaming, a model catalog DB synced from Hugging Face, and a Makepad front end using Rust channels for async UI updates. MVP ships after weeks of work, downloads models, streams tokens, and works but shows variable accuracy and speed depending on model size.
AI-written summary · about
chapters (9)
- 0:00Introduction to Moly
- 2:58Demo of Moly's interface
- 14:23Open source and architecture
- 16:11Why Rust for Moly
- 22:38Frontend Rust framework
- 30:15UI development with Maelstrom
- 40:55Prompt engineering demo
- 51:11Roadmap and future plans
- 55:52Call to action and Q&A
related talks
39mMoxin: The All-In-One Tool to Run LLMs on Your Personal Device
55mFast And Portable LLM Inference With WebAssembly And Rust
17mBuilding a self-hosted LLM ecosystem in Rust
40mOpen Source LLMs in the Cloud: Scalable Solutions
16mReact + WebLLM: Building Intelligent Applications with In-Browser Language Models
30mP2P Distributed LLM Inference Across Browsers With llama.cpp And WebRTC
34mPrivacy First: Building LLM-Powered Web Apps with client side WASM
27mWasm Is Becoming the Runtime for LLMs