Wrath of Math | Least Squares Solutions and Deriving the Normal Equation | Linear Algebra @WrathofMath | Uploaded 2 months ago | Updated 2 hours ago
We introduce the least squares problem and how to solve it using the techniques of linear algebra. We'll discuss least squares error and the least squares error vector. The idea is that we are finding the best approximation, in a subspace W, of a vector u that does not lie in the subspace. We'll find that the vector we seek is the projection of u onto the subspace W. This is the best approximation theorem, which we prove. We'll see how it applies in R^n to solve the least squares problem for systems of linear equations, and find it leads to something called the normal equation for a matrix. By using the normal equation, we'll find the least squares solution for two inconsistent systems Ax=b. Our first example has a unique least squares solution and our second example has infinitely many least squares solutions, which we will represent parametrically. #linearalgebra
Pythagorean Theorem in Inner Product Spaces: youtu.be/ncfScVfsLm8
Linear Algebra course: youtube.com/playlist?list=PLztBpqftvzxWT5z53AxSqkSaWDhAeToDG
Linear Algebra exercises: youtube.com/playlist?list=PLztBpqftvzxVmiiFW7KtPwBpnHNkTVeJc
Join Wrath of Math to get exclusive videos, music, and more:
youtube.com/channel/UCyEKvaxi8mt9FMc62MHcliw/join
0:00 Intro
0:53 An Inconsistent System and Why to Solve It
2:43 Least Squares Solutions and Least Squares Error
3:57 Why is it "Least Squares"?
5:27 Seeing the Solution
9:28 Best Approximation Theorem in Inner Product Spaces
13:21 Best Approximation Theorem in R^n
14:34 Deriving the Normal Equation
17:00 Consistency of the Normal Equation
18:21 Full Least Squares Example (Unique Solution)
21:06 Full Least Squares Example (Infinitely Many Solutions)
23:40 Conclusion
◉Textbooks I Like◉
Graph Theory: amzn.to/3JHQtZj
Real Analysis: amzn.to/3CMdgjI
Abstract Algebra: amzn.to/3IjoZaO
Linear Algebra: amzn.to/43xAWEz
Calculus: amzn.to/3PieD1M
Proofs and Set Theory: amzn.to/367VBXP (available for free online)
Statistics: amzn.to/3tsaEER
Discrete Math: amzn.to/3qfhoUn
Number Theory: amzn.to/3JqpOQd
★DONATE★
◆ Support Wrath of Math on Patreon for early access to new videos and other exclusive benefits: patreon.com/join/wrathofmathlessons
◆ Donate on PayPal: paypal.me/wrathofmath
Thanks to Loke Tan, Raül Beienheimer, Matt Venia, Micheline, Doug Walker, Odd Hultberg, Marc, Shlome Ashkenazi, Barbora Sharrock, Mohamad Nossier, Rolf Waefler, Shadow Master, and James Mead for their generous support on Patreon!
Outro music is mine. You cannot find it anywhere, for now.
Follow Wrath of Math on...
● Instagram: instagram.com/wrathofmathedu
● Facebook: facebook.com/WrathofMath
● Twitter: twitter.com/wrathofmathedu
My Math Rap channel: youtube.com/channel/UCQ2UBhg5nwWCL2aPC7_IpDQ/featured
We introduce the least squares problem and how to solve it using the techniques of linear algebra. We'll discuss least squares error and the least squares error vector. The idea is that we are finding the best approximation, in a subspace W, of a vector u that does not lie in the subspace. We'll find that the vector we seek is the projection of u onto the subspace W. This is the best approximation theorem, which we prove. We'll see how it applies in R^n to solve the least squares problem for systems of linear equations, and find it leads to something called the normal equation for a matrix. By using the normal equation, we'll find the least squares solution for two inconsistent systems Ax=b. Our first example has a unique least squares solution and our second example has infinitely many least squares solutions, which we will represent parametrically. #linearalgebra
Pythagorean Theorem in Inner Product Spaces: youtu.be/ncfScVfsLm8
Linear Algebra course: youtube.com/playlist?list=PLztBpqftvzxWT5z53AxSqkSaWDhAeToDG
Linear Algebra exercises: youtube.com/playlist?list=PLztBpqftvzxVmiiFW7KtPwBpnHNkTVeJc
Join Wrath of Math to get exclusive videos, music, and more:
youtube.com/channel/UCyEKvaxi8mt9FMc62MHcliw/join
0:00 Intro
0:53 An Inconsistent System and Why to Solve It
2:43 Least Squares Solutions and Least Squares Error
3:57 Why is it "Least Squares"?
5:27 Seeing the Solution
9:28 Best Approximation Theorem in Inner Product Spaces
13:21 Best Approximation Theorem in R^n
14:34 Deriving the Normal Equation
17:00 Consistency of the Normal Equation
18:21 Full Least Squares Example (Unique Solution)
21:06 Full Least Squares Example (Infinitely Many Solutions)
23:40 Conclusion
◉Textbooks I Like◉
Graph Theory: amzn.to/3JHQtZj
Real Analysis: amzn.to/3CMdgjI
Abstract Algebra: amzn.to/3IjoZaO
Linear Algebra: amzn.to/43xAWEz
Calculus: amzn.to/3PieD1M
Proofs and Set Theory: amzn.to/367VBXP (available for free online)
Statistics: amzn.to/3tsaEER
Discrete Math: amzn.to/3qfhoUn
Number Theory: amzn.to/3JqpOQd
★DONATE★
◆ Support Wrath of Math on Patreon for early access to new videos and other exclusive benefits: patreon.com/join/wrathofmathlessons
◆ Donate on PayPal: paypal.me/wrathofmath
Thanks to Loke Tan, Raül Beienheimer, Matt Venia, Micheline, Doug Walker, Odd Hultberg, Marc, Shlome Ashkenazi, Barbora Sharrock, Mohamad Nossier, Rolf Waefler, Shadow Master, and James Mead for their generous support on Patreon!
Outro music is mine. You cannot find it anywhere, for now.
Follow Wrath of Math on...
● Instagram: instagram.com/wrathofmathedu
● Facebook: facebook.com/WrathofMath
● Twitter: twitter.com/wrathofmathedu
My Math Rap channel: youtube.com/channel/UCQ2UBhg5nwWCL2aPC7_IpDQ/featured
