Uploaded October 2024 | Updated November 2026, 5 days ago
Why is such a simple equation so difficult to solve? Head to kiwico.com/welchlabs and use code WELCHLABS for 50% off your first monthly club crate or 20% off your first Panda Crate.
Welch Labs Imaginary Numbers Book!
welchlabs.com/resources/imaginary-numbers-book
Welch Labs Posters: welchlabs.com/resources
How the Bizarre Path of Mars Reshaped Astronomy: youtu.be/Phscjl0u6TI
Support Welch Labs on Patreon! patreon.com/welchlabs
Special thanks to Patrons: Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann
Learn more about WelchLabs!
welchlabs.com
TikTok: tiktok.com/@welchlabs
Instagram: instagram.com/welchlabs
REFERENCES
Colwell, P. (1993). Solving Kepler's Equation Over Three Centuries. United Kingdom: Willmann-Bell.
Needham, T. (1997). Visual Complex Analysis. United Kingdom: Clarendon Press.
Bate, R. R., Mueller, D. D., White, J. E. (1971). Fundamentals of Astrodynamics. Egypt: Dover Publications.
Vallado, D. (2001). Fundamentals of Astrodynamics and Applications. Netherlands: Springer Netherlands.
Borghi R. On the Bessel Solution of Kepler’s Equation. *Mathematics*. 2024; 12(1):154. doi.org/10.3390/math12010154
Tom Archibald, Craig Fraser, Ivor Grattan-Guinness, The History of Differential Equations, 1670–1950. Oberwolfach Rep. 1 (2004), no. 4, pp. 2729–2794
Francisco G. M. Orlando, C. Farina, Carlos A. D. Zarro, P. Terra; Kepler's equation and some of its pearls. Am. J. Phys. 1 November 2018; 86 (11): 849–858.
Arthur A. Rambaut, M.A. A Simple Method of obtaining an Approximate Solution of Kepler's Equation. *Monthly Notices of the Royal Astronomical Society*, Volume 50, Issue 5, March 1890, Pages 301–302.
Ben Coleman. How to Find the Taylor Series of an Inverse Function. randorithms.com/2021/08/31/Taylor-Series-Inverse.html
17 Recent papers on Kepler’s equation can be found in references 2-16 here: mdpi.com/2227-7390/12/1/154
CFAQJOTYQHT7JYIT
Why is such a simple equation so difficult to solve? Head to kiwico.com/welchlabs and use code WELCHLABS for 50% off your first monthly club crate or 20% off your first Panda Crate.
Welch Labs Imaginary Numbers Book!
welchlabs.com/resources/imaginary-numbers-book
Welch Labs Posters: welchlabs.com/resources
How the Bizarre Path of Mars Reshaped Astronomy: youtu.be/Phscjl0u6TI
Support Welch Labs on Patreon! patreon.com/welchlabs
Special thanks to Patrons: Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann
Learn more about WelchLabs!
welchlabs.com
TikTok: tiktok.com/@welchlabs
Instagram: instagram.com/welchlabs
REFERENCES
Colwell, P. (1993). Solving Kepler's Equation Over Three Centuries. United Kingdom: Willmann-Bell.
Needham, T. (1997). Visual Complex Analysis. United Kingdom: Clarendon Press.
Bate, R. R., Mueller, D. D., White, J. E. (1971). Fundamentals of Astrodynamics. Egypt: Dover Publications.
Vallado, D. (2001). Fundamentals of Astrodynamics and Applications. Netherlands: Springer Netherlands.
Borghi R. On the Bessel Solution of Kepler’s Equation. *Mathematics*. 2024; 12(1):154. doi.org/10.3390/math12010154
Tom Archibald, Craig Fraser, Ivor Grattan-Guinness, The History of Differential Equations, 1670–1950. Oberwolfach Rep. 1 (2004), no. 4, pp. 2729–2794
Francisco G. M. Orlando, C. Farina, Carlos A. D. Zarro, P. Terra; Kepler's equation and some of its pearls. Am. J. Phys. 1 November 2018; 86 (11): 849–858.
Arthur A. Rambaut, M.A. A Simple Method of obtaining an Approximate Solution of Kepler's Equation. *Monthly Notices of the Royal Astronomical Society*, Volume 50, Issue 5, March 1890, Pages 301–302.
Ben Coleman. How to Find the Taylor Series of an Inverse Function. randorithms.com/2021/08/31/Taylor-Series-Inverse.html
17 Recent papers on Kepler’s equation can be found in references 2-16 here: mdpi.com/2227-7390/12/1/154
CFAQJOTYQHT7JYIT


![Yann LeCuns $1B Bet Against LLMs [Part 1]
Apply to join Hudson River Trading: https://www.hudsonrivertrading.com/welchlabs
Welch Labs Book: https://www.welchlabs.com/resources/ai-book-ezrzm-msrmc
Patreon: https://www.patreon.com/c/welchlabs
Part 2: https://www.youtube.com/watch?v=v_jDvpEGTIg
Sections
0:00 - Intro
2:28 - The Problem with Deep Learning
4:17 - Intelligence is a Cake
5:15 - The Rise of Generative AI
8:00 - Blurry Images
8:54 - HRT is an awesome place to work
11:16 - But why so Blurry?
13:30 - Do our models need to be generative?
15:16 - Siamese Networks
17:53 - Representation Collapse
19:54 - Yann’s Epiphany & Barlow Twins
27:22 - DINO
28:58 - JEPA & World Models
34:09 - But is JEPA good?
36:19 - Welch Labs Book
Special thanks to: Yann LeCun, Stephane Deny, David Fan, Nicolas Ballas
Clip of Yann from 1989: https://www.youtube.com/watch?v=FwFduRA_L6Q
CNN Paper: http://yann.lecun.com/exdb/publis/pdf/lecun-89e.pdf
LeNet-5 paper: http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf
Dashcam video
https://commons.wikimedia.org/wiki/File:Car_Driving_Faadou_4K_HDR Rural_road Canton 327.webm
Image Credits
https://en.wikipedia.org/wiki/File:Dota_2_Gameplay_Aug_2017.jpg
https://commons.wikimedia.org/wiki/File:Felis_catus-cat_on_snow.jpg
https://commons.wikimedia.org/wiki/File:Magnificent_CME_Erupts_on_the_Sun August_31.jpg
https://commons.wikimedia.org/wiki/File:Alcedo_atthis Riserve_naturali_e_aree_contigue_della_fascia_fluviale_del_Po.jpg
https://commons.wikimedia.org/wiki/File:Biandintz_eta_zaldiak modified2.jpg
V-JEPA2 Robot Arm Videos
https://ai.meta.com/research/vjepa/
PATRONS
Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann, Jason Singh, Robert Riley, vornska, Barry Silverman, Jake Ehrlich, Mitch Jacobs, Lauren Steely, Jeff Eastman, Rodolfo Ibarra, Clark Barrus, Rob Napier, Andrew White, Richard B Johnston, abhiteja mandava, Burt Humburg, Kevin Mitchell, Daniel Sanchez, Ferdie Wang, Tripp Hill, Richard Harbaugh Jr, Prasad Raje, Kalle Aaltonen, Midori Switch Hound, Zach Wilson, Chris Seltzer, Ven Popov, Hunter Nelson, Amit Bueno, Scott Olsen, Johan Rimez, Shehryar Saroya, Tyler Christensen, Beckett Madden-Woods, Darrell Thomas, Javier Soto, U007D, Caleb Begly, Rick Rubenstein, Brent Hunsaker, Dan Patterson, Tchsurvives, Alex Adai, Walter Reade, Zyansheep, Walter Reade, Duncan Stannett, Reginald Carey, Jean-Manuel Izaret, dh71633, Adrian Rodriguez, Dimitar Stojanovski, Michael Harder, Peter Maldonado, Emily Pesce, David Johnston, Insang Song, FaeTheWolf, Stephen Taylor, KittenKaboodle, EMatter, PATRICKMCCORMACK, John Beahan, Cameron, Cole Jones, Garrett Thornburg, Jeroen W, Rohit Sharma, GlennB, Emmanuel Cortes, Katie Quinn, Karina C, Cakra WW, Mike Ton, Eric Gometz, MacCallister Higgins, Niko Drossos, David Eraso, Tom Zehle, Steve, Brian Lineburg, rjbl, Michael Loh, Perry Vais, Bengal0, Farhad Manjoo, Sara Chipps, Ellis Driscoll, William Taysom, Will Harmon, CK, Abdullah, Peter Cho, Leo Nikora, Griffin Smith, Ash Katnoria, Alex, Markus Hays Nielsen, Catherine H., Vi, David Dobáš, Peter Wang, Sina Sohangir, Danny Thomas, Julian Francis, Hans Adler, Jiayu Peng, Weston M, Youssouf da Silva, John Thomas, Samuel Costello, Sam Adams, Bryan Liles, Malaya Zemlya, Karl, Vahe Andonians, Mike Doughty, Larry Novelo, Jonas Acres, Ludicrum Rex, Robert Blumofe, Anthony Z, Alex Zhao, Dan Babitch, Nikko Patten
Supporting code: https://github.com/WelchLabs/videos
Created by: Sam Baskin, Pranav Gundu, and Stephen Welch
Content ID: CFAQJOTYQHT7JYIT Yann LeCuns $1B Bet Against LLMs [Part 1]](https://i.ytimg.com/vi/kYkIdXwW2AE/mqdefault.jpg)

![ChatGPT is made from 100 million of these [The Perceptron]
Go to https://drinkag1.com/welchlabs to subscribe and save $20 off your first subscription of AG1! Thanks to AG1 for sponsoring todays video.
Imaginary Numbers book is back in stock! Update at 23:11
https://www.welchlabs.com/resources/imaginary-numbers-book
Welch Labs Posters:
https://www.welchlabs.com/resources
Perceptrons designs from viewers:
https://sites.google.com/view/jacklangsdorf-funprojects/home/perceptron?authuser=0
https://github.com/rdb64-hobbies/Perceptron/blob/main/BUILDING.md
Really nice Perceptron simulators built by viewers:
https://github.com/srives/Perceptron
https://priyangsubanerjee.github.io/perceptron-simulator/
Special Thanks to Patrons https://www.patreon.com/welchlabs
Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann, Jason Singh, Robert Riley, vornska, Barry Silverman
References
Rumelhart, D. E., Mcclelland, J. L. (1987). Parallel Distributed Processing, Volume 1: Explorations in the Microstructure of Cognition: Foundations. United Kingdom: Penguin Random House LLC.
Talking Nets: An Oral History of Neural Networks. (2000). United Kingdom: MIT Press.
Prince, S. J. (2023). Understanding Deep Learning. United Kingdom: MIT Press.
Crevier, D. (1993). AI : the tumultuous history of the search for artificial intelligence. New York: Basic Books.
Cat and dog face dataset: https://www.kaggle.com/datasets/andrewmvd/animal-faces?resource=download
Minsky, M., Papert, S. (2017). Perceptrons: An Introduction to Computational Geometry. United Kingdom: MIT Press.
Widrow, Bernard, and Michael A. Lehr. 30 years of adaptive neural networks: perceptron, madaline, and backpropagation. *Proceedings of the IEEE* 78.9 (1990): 1415-1442.
Olazaran, Mikel. A sociological history of the neural network controversy. *Advances in computers*. Vol. 37. Elsevier, 1993. 335-425.
Widrow, Bernard. Generalization and information storage in networks of adaline neurons. *Self-organizing systems* (1962): 435-461.
Widrow, Bernard. Thinking about thinking: the discovery of the LMS algorithm. *IEEE Signal Processing Magazine* 22.1 (2005): 100-106.
Technical Notes
Method for counting neurons in ChatGPT: Starting with GPT-2 implementation here: https://github.com/karpathy/build-nanogpt/blob/master/train_gpt2.py - keys, queries, and values are implemented in Linear layers with n_embd inputs and 3*n_embd outputs, where n_embd is the embedding dimension. Output projection layer has n_embd and n_embd outputs. So a single attention layer will have ~4*n_embd neurons. GPT-3 has an embedding dimension of 12,288, so each attention layer has ~49,152 neurons. Each MLP block has n_embd inputs, 4*n_embd hidden units, and n_embd outputs, so ~5*n_embd total neurons, or ~61,440. Total neuron count for GPT-3 is then 96*(49,152+61,440)=10,616,832, ignoring initial embedding and final unembedding. Finally, GPT-4 reportedly has ~1.8 Trillion parameters (https://semianalysis.com/2023/07/10/gpt-4-architecture-infrastructure/), making it ~10x larger than GPT-3. Note that GPT-4 is reportedly a mixture of experts, and not all experts are used for each inference, so it appears that not all 1.8 trillion parameters are used for a given inference call. Assuming that ~10x the parameters means 10x the neurons, then GPT-4 should have ~100M neurons.
CFAQJOTYQHT7JYIT ChatGPT is made from 100 million of these [The Perceptron]](https://i.ytimg.com/vi/l-9ALe3U-Fg/mqdefault.jpg)
![Waffles And Harmonic Motion [Part 1]
Waffles And Harmonic Motion [Part 1] Waffles And Harmonic Motion [Part 1]](https://i.ytimg.com/vi/nKavBl1EWiI/mqdefault.jpg)




![Why Deep Learning Works Unreasonably Well [How Models Learn Part 3]
Take your personal data back with Incogni! Use code WELCHLABS and get 60% off an annual plan: http://incogni.com/welchlabs
Welch Labs Guide to AI: https://www.welchlabs.com/resources/ai-book-ezrzm-msrmc
New Patreon Rewards 33:31- own a piece of Welch Labs history!
https://www.patreon.com/welchlabs
Books & Posters
https://www.welchlabs.com/resources
Sections
0:00 - Intro
4:49 - How Incogni Saves Me Time
6:32 - Part 2 Recap
8:10 - Moving to Two Layers
9:15 - How Activation Functions Fold Space
11:45 - Numerical Walkthrough
13:42 - Universal Approximation Theorem
15:45 - The Geometry of Backpropagation
19:52 - The Geometry of Depth
24:27 - Exponentially Better?
30:23 - Neural Networks Demystifed
31:50 - The Time I Quit YouTube
33:31 - New Patreon Rewards!
Special Thanks to Patrons https://www.patreon.com/welchlabs
Juan Benet, Ross Hanson, Yan Babitski, AJ Englehardt, Alvin Khaled, Eduardo Barraza, Hitoshi Yamauchi, Jaewon Jung, Mrgoodlight, Shinichi Hayashi, Sid Sarasvati, Dominic Beaumont, Shannon Prater, Ubiquity Ventures, Matias Forti, Brian Henry, Tim Palade, Petar Vecutin, Nicolas baumann, Jason Singh, Robert Riley, vornska, Barry Silverman, Jake Ehrlich, Mitch Jacobs, Lauren Steely, Jeff Eastman, Rodolfo Ibarra, Clark Barrus, Rob Napier, Andrew White, Richard B Johnston, abhiteja mandava, Burt Humburg, Kevin Mitchell, Daniel Sanchez, Ferdie Wang, Tripp Hill, Richard Harbaugh Jr, Prasad Raje, Kalle Aaltonen, Midori Switch Hound, Zach Wilson, Chris Seltzer, Ven Popov, Hunter Nelson, Amit Bueno, Scott Olsen, Johan Rimez, Shehryar Saroya, Tyler Christensen, Beckett Madden-Woods, Darrell Thomas, Javier Soto
References
Simon Prince, Understanding Deep Learning. https://udlbook.github.io/udlbook/
Liang, Shiyu, and Rayadurgam Srikant. Why deep neural networks for function approximation?. arXiv preprint arXiv:1610.04161 (2016).
Hanin, Boris, and David Rolnick. Deep relu networks have surprisingly few activation patterns. *Advances in neural information processing systems* 32 (2019).
Hanin, Boris, and David Rolnick. Complexity of linear regions in deep networks. *International Conference on Machine Learning*. PMLR, 2019.
Fan, Feng-Lei, et al. Deep relu networks have surprisingly simple polytopes. *arXiv preprint arXiv:2305.09145* (2023).
All Code:
https://github.com/stephencwelch/manim_videos
100k neuron wide example training code: https://github.com/stephencwelch/manim_videos/blob/master/_2025/backprop_3/notebooks/Wide%20Training%20Example.ipynb
Code from viewer Hugo Brouwer that achieves 99.5%+ with less than 100 neurons using Fourier Features!
https://github.com/AgntBrwr/baarle-hertog-fourier-features
Code from viewer Nico Waser that uses 100 neurons:
https://github.com/Waser2004/Illustrated Guide_to_AI/tree/main/Chapter%204%20-%20Deep%20Learning
Written by: Stephen Welch
Produced by: Stephen Welch, Sam Baskin, and Pranav Gundu
Premium Beat IDs
EEDYZ3FP44YX8OWTe
MWROXNAY0SPXCMBS
CFAQJOTYQHT7JYIT Why Deep Learning Works Unreasonably Well [How Models Learn Part 3]](https://i.ytimg.com/vi/qx7hirqgfuU/mqdefault.jpg)