Apropos DeepSeek
summary
DeepSeek releases an open model (R1 under MIT license) that claims real gains: a 671 billion-parameter base with 37 expert activations and training on “two tera tokens” (Common Crawl) plus a math-focused post-training that uses MoE routing and a reinforcement-learning-only run that invents its own terminology. Key engineering moves are multi-head latent attention that compresses key/value cache and a Mixture of Experts design to lower memory per token. Papers and code are public, but GPU-count/training-cost claims and data provenance remain ambiguous, and deployment shows political filtering and unencrypted app telemetry.
AI-written summary · about
chapters (6)
- 0:00DeepSeek's impact
- 2:48Large language models basics
- 12:43Model efficiency innovations
- 18:06Multi-head latent attention & mixture of experts
- 28:43Security to geopolitics concerns
- 42:25Innovation & security implications
related talks
12mDeepseek is Jambavanth moment for India
29mAligning DeepSeek R1
54mThe Future of AI: Building the Most Impactful Technology Together
06mHere's why people are not happy with DeepSeek
34mMapping the Open Model Landscape in 2025
25mProtecting AI Infrastructure
16mNous Research
43mHarnessing DeepSeek-R1 and OpenSearch for RAG-Powered Intelligent Info...