@3blue1brown

@3blue1brown

3Blue1Brown | What is backpropagation really doing? | Chapter 3, Deep learning @3blue1brown | Uploaded 6 years ago | Updated 19 minutes ago
What's actually happening to a neural network as it learns?
Help fund future projects: patreon.com/3blue1brown
An equally valuable form of support is to simply share some of the videos.
Special thanks to these supporters: http://3b1b.co/nn3-thanks
Written/interactive form of this series: 3blue1brown.com/topics/neural-networks

And by CrowdFlower: http://3b1b.co/crowdflower
Home page: 3blue1brown.com

The following video is sort of an appendix to this one. The main goal with the follow-on video is to show the connection between the visual walkthrough here, and the representation of these "nudges" in terms of partial derivatives that you will find when reading about backpropagation in other resources, like Michael Nielsen's book or Chis Olah's blog.

Video timeline:
0:00 - Introduction
0:23 - Recap
3:07 - Intuitive walkthrough example
9:33 - Stochastic gradient descent
12:28 - Final words

Thanks to these viewers for their contributions to translations
Italian: @teobucci
Vietnamese: CyanGuy111

What is backpropagation really doing? | Chapter 3, Deep learning

The determinant | Chapter 6, Essence of linear algebra

Snells law proof using springs

Positioned as the hardest question on a Putnam exam (#6, 1992)

Tattoos on Math

The surface area of a sphere

$But why would light slow down? | Optics puzzles 3 How the index of refraction arises, and why it depends on color. Quotebook Notebooks: https://3b1b.co/store These lessons are primarily funded directly by viewers: https://3b1b.co/support An equally valuable form of support is to simply share the videos. Looking Glass Universe videos on the index of refraction: https://youtu.be/uo3ds0FVpXs Much of this video is based on the following Feynmann lecture https://www.feynmanlectures.caltech.edu/I_31.html The explanation for why the phase of a wave produced by a plane of oscillating charges is a quarter phase behind the wave of a charge in the center of that plane, and hence a quarter phase behind that of a light wave inducing the oscillations, is given in the previous chapter: https://www.feynmanlectures.caltech.edu/I_30.html Sections: 0:00 - The standard explanation 3:14 - The plan 5:09 - Phase kicks 8:25 - What causes light? 13:20 - Adding waves 16:40 - Modeling the charge oscillation 20:59 - The driven harmonic oscillator 26:57 - End notes Thanks to these viewers for their contributions to translations German: Ole Hebrew: Alon Altman, Omer Tuchfeld Hindi: Saurabh Portuguese: @doryhelioaires Urdu: Momini0 These animations are largely made using a custom Python library, manim. See the FAQ comments here: https://3b1b.co/faq#manim https://github.com/3b1b/manim https://github.com/ManimCommunity/manim/ All code for specific videos is visible here: https://github.com/3b1b/videos/ The music is by Vincent Rubinetti. https://www.vincentrubinetti.com https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u 3blue1brown is a channel about animating math, in all senses of the word animate. If youre reading the bottom of a video description, Im guessing youre more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly. Mailing list: https://3blue1brown.substack.com Twitter: https://twitter.com/3blue1brown Instagram: https://www.instagram.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Facebook: https://www.facebook.com/3blue1brown Patreon: https://patreon.com/3blue1brown Website: https://www.3blue1brown.com$

Exponential growth and epidemics

But what is a convolution?

$Beyond the Mandelbrot set, an intro to holomorphic dynamics An intro to holomorphic dynamics, the study of iterated complex functions. Video on Newtons fractal: https://youtu.be/-RdOwhmqP5s Special thanks to these supporters: https://3b1b.co/lessons/holomorphic-dynamics#thanks Extra special thanks to Sergey Shemyakov, of Aix-Marseille University, for helpful conversations and for introducing me to this phenomenon. Introduction to Fatou sets and Julia sets, including a discussion of Montels theorem and its consequences: http://www.math.stonybrook.edu/~scott/Papers/India/Fatou-Julia.pdf Numberphile with Ben Sparks on the Mandelbrot set: https://youtu.be/FFftmWSzgmk Ben explains how he made the Geogebra files on his channel here: https://youtu.be/ICqj7nJbiRI (part 1) https://youtu.be/3BXoNOahFJk (part 2) Excellent article on Acko.net, from the basics of building up complex numbers to Julia sets. https://acko.net/blog/how-to-fold-a-julia-fractal/ Bit of a side note, but if you want an exceedingly beautiful rendering of the quaternion-version of Julia fractals, take a look at this Inigo Quilez video: https://www.youtube.com/watch?v=rQ2bnU4dkso I first saw Fatous theorem in this article: https://projecteuclid.org/journals/communications-in-mathematical-physics/volume-91/issue-2/On-the-iteration-of-a-rational-function computer-experiments/cmp/1103940533.pdf Moduli spaces of Newton maps: https://arxiv.org/pdf/1512.05098.pdf On Montels theorem: https://people.ucsc.edu/~fmonard/Sp17_Math207/lecture11.pdf On Newtons Fractal: https://core.ac.uk/download/pdf/1633889.pdf Thanks to these viewers for their contributions to translations Hebrew: Omer Tuchfeld These animations are largely made using a custom python library, manim. See the FAQ comments here: https://www.3blue1brown.com/faq#manim https://github.com/3b1b/manim https://github.com/ManimCommunity/manim/ You can find code for specific videos and projects here: https://github.com/3b1b/videos/ Music by Vincent Rubinetti. https://www.vincentrubinetti.com/ Download the music on Bandcamp: https://vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown Stream the music on Spotify: https://open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u Timestamps: 0:00 - Intro 3:02 - Rational functions 4:15 - The Mandelbrot set 8:12 - Fixed points and stability 12:51 - Cycles 16:25 - Hidden Mandelbrot 21:17 - Fatou sets and Julia sets 26:24 - Final thoughts 3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe: http://3b1b.co/subscribe Various social media stuffs: Website: https://www.3blue1brown.com Twitter: https://twitter.com/3blue1brown Reddit: https://www.reddit.com/r/3blue1brown Instagram: https://www.instagram.com/3blue1brown_animations/ Patreon: https://patreon.com/3blue1brown Facebook: https://www.facebook.com/3blue1brown$

Dot products and duality | Chapter 9, Essence of linear algebra

The unexpectedly hard windmill question (2011 IMO, Q2)

What is backpropagation really doing? | Chapter 3, Deep learning @3blue1brown