Uploaded August 2022 | Updated September 2026, 3 weeks ago

---

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.

Support my work (and get early access to new videos and source code) on Patreon or Nebula
* Patreon: patreon.com/SebastianLague
* Nebula: go.nebula.tv/sebastianlague

Source code and Demo:
* GitHub: github.com/SebLague/Neural-Network-Experiments
* Demo: sebastian.itch.io/neural-network-experiment

Training Data:
* Digits: yann.lecun.com/exdb/mnist
* Fashion: github.com/zalandoresearch/fashion-mnist
* Doodles: github.com/googlecreativelab/quickdraw-dataset
* Cifar10: https://www.cs.toronto.edu/~kriz/cifar.html

Chapters
0:00 Introduction
2:39 The decision boundary
3:49 Weights
5:42 Biases
6:45 Hidden layers
7:45 Programming the network
9:57 Activation functions
12:42 Cost
15:07 Gradient descent example
18:22 The cost landscape
19:55 Programming gradient descent
21:10 It's learning! (slowly)
23:21 Calculus example
27:34 The chain rule
29:50 Some partial derivatives
33:14 Backpropagation
39:25 Digit recognition
43:56 Drawing our own digits
47:37 Fashion
48:25 Doodles
52:00 The final challenge

Music:
Cosmic Waves - Michael FK
Amber - The Stolen Orchestra
Beyond the Horizon - Sounds Like Sander
Air - Assaf Ayalon
Purest Form - Sounds Like Sander
Hear Wide Open - Sounds Like Sander
Universal Wonder - Moments
Roman P - Moments
All In Good Time - Shimmer
It Will Come Back - The Stolen Orchestra
Frontier - Shimmer
New Moon - Cloud Wave
Sunflower - Cody Martin
Inner Peace - Moments
Enchanted - Cody Martin
Just Around The Corner - Shimmer