Mathemaniac | The weirdest paradox in statistics (and machine learning) @mathemaniac | Uploaded 2 years ago | Updated 2 hours ago
đ AD: Get Exclusive NordVPN deal here ⌠nordvpn.com/mathemaniac. It's risk-free with Nord's 30-day money-back guarantee! â
Second channel video: youtube.com/watch?v=3ne9yghOtw8
Stein's paradox is of fundamental importance in modern statistics, introducing concepts of shrinkage to further reduce the mean squared error, especially in higher dimensional statistics that is particularly relevant nowadays, in the world of machine learning, for example. However, this is usually ignored, because it is mostly seen as a toy problem. Precisely because it is such a simple problem that illustrates the problem of maximum likelihood estimation! This paradox is the subject of many blogposts (linked below), but not really here on YouTube, except in some lecture recordings, so I have to bring this up to YouTube.
This is not to say that maximum likelihood estimator is not useful - in most situations, especially in lower dimensional statistics, it is still good, but to hold it to such a high place, as statisticians did before 1961? That is not a healthy attitude to this theory.
One thing I did not say, but perhaps a lot of people will want me to, is that this is an emprical Bayes estimator, but again, more links below.
Video chapters:
00:00 Introduction
04:38 Chapter 1: The "best" estimator
09:48 Chapter 2: Why shrinkage works
15:51 Chapter 3: Bias-variance tradeoff
18:45 Chapter 4: Applications
Further reading:
The âbaseball paperâ: https://efron.ckirby.su.domains//other/Article1977.pdf
Wikipedia: en.wikipedia.org/wiki/Stein%27s_example
Dominating the (positive-part) James-Stein estimator: projecteuclid.org/journals/annals-of-statistics/volume-22/issue-3/Improving-on-the-James-Stein-Positive-Part-Estimator/10.1214/aos/1176325640.full
Wikipedia (Empirical Bayes): en.wikipedia.org/wiki/Empirical_Bayes_method
Other writeups:
http://www.ime.unicamp.br/~veronica/MI677/steinparadox.pdf
joe-antognini.github.io/machine-learning/steins-paradox
jchau.org/2021/01/29/demystifying-stein-s-paradox
naftaliharris.com/blog/steinviz
austinrochford.com/posts/2013-11-30-steins-paradox-and-empirical-bayes.html
duphan.wordpress.com/2016/07/10/steins-paradox-or-the-power-of-shrinkage
http://www.statslab.cam.ac.uk/~rjs57/SteinParadox.pdf
(Philosophical implications) http://philsci-archive.pitt.edu/13303/1/Philosophical%20significance%20of%20Stein%20Preprint.pdf
Other than commenting on the video, you are very welcome to fill in a Google form linked below, which helps me make better videos by catering for your math levels:
https://forms.gle/QJ29hocF9uQAyZyH6
If you want to know more interesting Mathematics, stay tuned for the next video!
SUBSCRIBE and see you in the next video!
If you are wondering how I made all these videos, even though it is stylistically similar to 3Blue1Brown, I don't use his animation engine Manim, but I will probably reveal how I did it in a potential subscriber milestone, so do subscribe!
Social media:
Facebook: facebook.com/mathemaniacyt
Instagram: instagram.com/_mathemaniac_
Twitter: twitter.com/mathemaniacyt
Patreon: patreon.com/mathemaniac (support if you want to and can afford to!)
Merch: mathemaniac.myspreadshop.co.uk
Ko-fi: ko-fi.com/mathemaniac [for one-time support]
For my contact email, check my About page on a PC.
See you next time!
đ AD: Get Exclusive NordVPN deal here ⌠nordvpn.com/mathemaniac. It's risk-free with Nord's 30-day money-back guarantee! â
Second channel video: youtube.com/watch?v=3ne9yghOtw8
Stein's paradox is of fundamental importance in modern statistics, introducing concepts of shrinkage to further reduce the mean squared error, especially in higher dimensional statistics that is particularly relevant nowadays, in the world of machine learning, for example. However, this is usually ignored, because it is mostly seen as a toy problem. Precisely because it is such a simple problem that illustrates the problem of maximum likelihood estimation! This paradox is the subject of many blogposts (linked below), but not really here on YouTube, except in some lecture recordings, so I have to bring this up to YouTube.
This is not to say that maximum likelihood estimator is not useful - in most situations, especially in lower dimensional statistics, it is still good, but to hold it to such a high place, as statisticians did before 1961? That is not a healthy attitude to this theory.
One thing I did not say, but perhaps a lot of people will want me to, is that this is an emprical Bayes estimator, but again, more links below.
Video chapters:
00:00 Introduction
04:38 Chapter 1: The "best" estimator
09:48 Chapter 2: Why shrinkage works
15:51 Chapter 3: Bias-variance tradeoff
18:45 Chapter 4: Applications
Further reading:
The âbaseball paperâ: https://efron.ckirby.su.domains//other/Article1977.pdf
Wikipedia: en.wikipedia.org/wiki/Stein%27s_example
Dominating the (positive-part) James-Stein estimator: projecteuclid.org/journals/annals-of-statistics/volume-22/issue-3/Improving-on-the-James-Stein-Positive-Part-Estimator/10.1214/aos/1176325640.full
Wikipedia (Empirical Bayes): en.wikipedia.org/wiki/Empirical_Bayes_method
Other writeups:
http://www.ime.unicamp.br/~veronica/MI677/steinparadox.pdf
joe-antognini.github.io/machine-learning/steins-paradox
jchau.org/2021/01/29/demystifying-stein-s-paradox
naftaliharris.com/blog/steinviz
austinrochford.com/posts/2013-11-30-steins-paradox-and-empirical-bayes.html
duphan.wordpress.com/2016/07/10/steins-paradox-or-the-power-of-shrinkage
http://www.statslab.cam.ac.uk/~rjs57/SteinParadox.pdf
(Philosophical implications) http://philsci-archive.pitt.edu/13303/1/Philosophical%20significance%20of%20Stein%20Preprint.pdf
Other than commenting on the video, you are very welcome to fill in a Google form linked below, which helps me make better videos by catering for your math levels:
https://forms.gle/QJ29hocF9uQAyZyH6
If you want to know more interesting Mathematics, stay tuned for the next video!
SUBSCRIBE and see you in the next video!
If you are wondering how I made all these videos, even though it is stylistically similar to 3Blue1Brown, I don't use his animation engine Manim, but I will probably reveal how I did it in a potential subscriber milestone, so do subscribe!
Social media:
Facebook: facebook.com/mathemaniacyt
Instagram: instagram.com/_mathemaniac_
Twitter: twitter.com/mathemaniacyt
Patreon: patreon.com/mathemaniac (support if you want to and can afford to!)
Merch: mathemaniac.myspreadshop.co.uk
Ko-fi: ko-fi.com/mathemaniac [for one-time support]
For my contact email, check my About page on a PC.
See you next time!
