Emergent Trends in Large Multimodal Models
summary
A new encoder-free unified architecture called SenseNova U1 removes vision encoders and VAEs, projects patches with lightweight embeddings, and ships as a dense 8B+8B model and an MoE variant with ~3B active params. A curated spatial dataset SI8M (8 million examples) plus benchmarks show base models outperforming GPT‑5 and Gemini‑3 Pro across eight spatial tests. Video work demonstrates reasoning via diffusion denoising, reporting models 25% better than Sora 2 while humans still score 0.974 vs 0.685.
AI-written summary · about
chapters (5)
- 0:00Introduction to trends
- 1:57Unified architecture without encoders
- 4:46Spatial intelligence in 3D
- 7:56Video as next intelligence substrate
- 11:29Summary and future outlook
related talks
28mLarge Multimodal Models: Understanding the State of the Art in Compute
01h 23mMultimodal Deep Learning in the Real World
27mThe Cutting Edge of AI: From LLMs to Multimodal Intelligence
32mGenerative AI Text and Multimodal Embedding Models for Real World Use Cases
40mKeynote: Open-source Multimodal AI in the Wild
21mHow Google DeepMind is researching the next Frontier of AI for Gemini
41mBuilding Generative Image & Video models at Scale
39mNew multimodal vision AI models and their practical applications