Gradient Descent, Step-by-Step @statquest
Gradient Descent, Step-by-Step  @statquest
Uploaded February 2019 | Updated November 2026, 8 hours ago
Gradient Descent is the workhorse behind most of Machine Learning. When you fit a machine learning method to a training dataset, you're probably using Gradient Descent. It can optimize parameters in a wide variety of settings. Since it's so fundamental to Machine Learning, I decided to make a "step-by-step" video that shows you exactly how it works.

NOTE: This video assumes you are already familiar with Least Squares and Linear Regression. If not, here's the link to the Quest: youtu.be/PaFPbb66DxQ

Sources:
There are a ton of websites that describe the math behind Gradient Descent. One of my favorite is the wikipedia article: en.wikipedia.org/wiki/Gradient_descent

For a complete index of all the StatQuest videos, check out:
statquest.org/video-index

If you'd like to support StatQuest, please consider...

Patreon: patreon.com/statquest
...or...
YouTube Membership: youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
statquest.org/statquest-store

...or just donating to StatQuest!
paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
twitter.com/joshuastarmer

0:00 Awesome song and introduction
1:25 Main ideas behind Gradient Descent
5:38 Gradient Descent optimization of a single variable, part 1
9:08 An important note about why we use Gradient Descent
9:40 Gradient Descent optimization of a single variable, part 2
14:48 Review of concepts covered so far
15:48 Gradient Descent optimization of two (or more) variables
21:55 A note about Loss Functions
22:13 Gradient Descent algorithm
23:06 Stochastic Gradient Descent

#statquest #gradient #descent #ML
Gradient Descent, Step-by-StepWhy Dividing By N Underestimates the VarianceThe AI Buzz, Episode #4: ChatGPT + Bing and How to start an AI company in 3 easy steps.Happy DaysLive 2020-04-06!!! Naive Bayes: GaussianNaive Bayes: Concepts and CodeLinear Discriminant Analysis (LDA): Concepts and CodeStochastic Gradient Descent, Clearly Explained!!!Machine Learning Fundamentals: Sensitivity and Specificityp-values: What they are and how to interpret themWord Embedding and Word2Vec, Clearly Explained!!!Population and Estimated Parameters, Clearly Explained!!!
StatQuest with Josh Starmer |

Gradient Descent, Step-by-Step

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER