Let’s build K I T T with JavaScript
summary
Builds a fully offline Kit replica in the browser using JavaScript: local LLMs (Gemma 2 2B and smaller open models) plus transformers.js for on-device inference with WebGPU and WebAssembly, Whisper tiny for ASR and kakoro/kakorojs for TTS, and an IndexedDB vector store of 6,275 parsed scenes. Summaries required ~3.1M input tokens and 303k output tokens. Retrieval-augmented generation with function calling drives agentic decisions; WebBluetooth talks to an Arduino Nano RP2040 Lego car (motor + RGB) and the demo actually moves the car and avoids a cliff.
AI-written summary · about
chapters (25)
- 0:00Introduction to Kit
- 2:10Building Kit in JavaScript
- 3:14Machine learning in the browser
- 6:28Demo of speech pipeline
- 7:09Adding intelligence with LLMs
- 8:55Large vs small language models
- 10:40WebLM for browser-based LLMs
- 11:22Demo of local LLM response
- 12:45Prompt engineering for personality
- 15:41Storing Night Rider episodes
- 17:29Similarity search explained
- 21:56Summarizing scenes with LLMs
- 24:05Function calling with text output
- 25:31Retrieval augmented generation
- 26:11Demo of Kit with memory
- 30:03Giving Kit a physical body
- 31:06Web Bluetooth API overview
- 33:13Demo of AI-controlled light
- 34:16LEGO car with Arduino
- 35:23Demo of remote-controlled car
- 36:27AI-driven car navigation
- 37:31Full Kit implementation summary
- 38:15Why build AI in the browser
- 39:19Browser AI as an opt-in feature
- 40:02Q&A and closing
