Unraveling the Magic of Two Hot Trends: WebAssembly and Generative AI
Talkat WasmCon 2023 · on Wednesday, 2023-09-06· at 16:30· Grand Ballroom I, Hyatt Regency Bellevue, 900 Bellevue Way NE·
summary
Panel argues that WebAssembly makes low-latency, portable inference practical today: teams run fine-tuned small models (MPT, Llama2, Code Llama) inside WebAssembly sandboxes, create embeddings stored in vector DBs, and use retrieval augmentation to reduce hallucinations. Real demos include in-visit ER note summarization and a travel/shop assistant that calls plugins and on-device data; models show emergent behavior past ~6B parameters and contexts approaching 100k tokens. Firms deploy via Fastly/Fermyon-style runtimes to run anywhere with controlled permissions.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:26Panelist backgrounds
- 4:00Generative AI overview
- 6:47Edge computing with WebAssembly
- 9:36Generative AI breakthroughs
- 13:48Investment thesis for AI startups
- 16:20Fine-tuning models matter
- 18:26AI application architecture
- 21:15WebAssembly in AI stack
- 22:39Generative AI use cases
- 23:20Personalized AI assistants
- 27:52Healthcare patient summaries
- 31:30Q&A
related talks
37mAI/ML at the Extreme Edge with WebAssembly: A Path Forward
01h 04mDevelop, Build & Deploy Lightweight AI Services with WebAssembly
08mLightning Talk: Accelerate WebAssembly AI with Backend Graph Compliers on a Wide Variety of AI HW Accelerators
44mWWWAI: Constructing a Truly Open GenAI App with WebAssembly, WebGPU, and WebAI
32mEmpower WebAssembly ML Over Diverse AI Hardware Accelerators Outside the Browser
40mWebAssembly for AI Infra : A Lightweight, Fast, and Secure Alternative Approach
41mWebAssembly Based AI as a Service
39mWebAssembly-Powered Machine Learning