Unveiling the simple math that powers AI
summary
Explains that neural nets reduce to multiplying and adding across vectors and matrices, using floating‑point storage (IEEE single precision) and activation functions like ReLU; shows training with mean squared error and gradient descent; cites model scales — 175 billion parameters — and an example interaction requiring 52 trillion FLOPs; demonstrates practical optimizations such as 32-bit to 8-bit quantization that shrinks storage at the cost of precision and justifies GPU/TPU use for massive parallel vector/matrix math.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:52Math proficiency background
- 5:22Binary system in AI
- 9:39Floating point storage
- 13:15Neural network transformations
- 21:50Loss in model training
- 25:45Model size and parameters
- 31:50AI processors overview
- 33:13AI’s carbon footprint
- 37:31Q&A
related talks
18mAI Is Not Magic: The Physics, Math, and Hardware Behind It
44mMachine Learning Exposed
51mMachine Learning Exposed Deep and Reinforcement Learning
27mDemystifying Deep Neural Networks
43mThe Linear Algebra of Deep Learning
24mSuper Accessible No Math Intro to Neural Networks For Beginners
55mA Tour of Deep Learning With C++
58mStrong Generalization from Small Brains and No Training Data