Writing a Neural Net from Scratch
summary
Code a neural net from scratch in C# inside LINQPad with no libraries, implementing perceptrons, leaky ReLU and other activations, and train by stochastic gradient descent with a tunable learning rate. Builds a batched and online trainer, visualizer, and experiments: a single neuron learns NAND; a small network trains on MNIST (~50,000 training / 10,000 test) and reaches ~95% test accuracy (up to ~97% with tweaks). Covers data augmentation, pretraining, and why convolutional layers improve image tasks.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:23Traditional Programming
- 3:12Why not use a library
- 4:33Training to neural networks
- 5:49Perceptron
- 6:45State
- 7:00Transient State
- 7:12NAND Gate
- 7:28Basic NAND Gate
- 9:23Logistic Sigmoid
- 11:23Runoff
- 12:29Error
- 13:36Loss
- 14:35Minimize
- 21:09Gradient Descent
- 24:13Summary
- 32:45Neural Network Architecture
- 51:08Convolutional Network
- 54:38Optimal Learning Rate
related talks
57mWriting a Neural Net from Scratch
54mNeural networks by example
43mA Gentle Introduction to Neural Networks
43m(Deep) Learn You a Neural Net for Great Good!
26mMachine Learning in the Browser with Deep Neural Networks
25mIntroduction to Neural Networks
01h 50mNeural Networks: Walkthrough
01h 02mNeural Networks: A Primer