Uploaded February 2020 | Updated November 2026, 7 hours ago
slides: mlvu.github.io/lecture/31.ProbabilisticModels1.annotated.pdf
course materials: mlvu.github.io
Today, we discuss probabilistic approaches to Machine Learning. We start by reviewing the basics of probability theory, and then discuss some approaches to probabilistic classification, including the (naive) Bayes classifier and logistic regression, which is a combination of a linear model, a sigmoid function and a log loss.
We finish with a high level overview of information theory. This gives us the concepts of entropy, cross-entropy and Kullback-Leibler Divergence, which we will make frequent use of in the lectures to come. We tie everything together by showing that log loss can be rephrased in terms of cross-entropy (which is why it often goes by the name of cross-entropy loss).
slides: mlvu.github.io/lecture/31.ProbabilisticModels1.annotated.pdf
course materials: mlvu.github.io
Today, we discuss probabilistic approaches to Machine Learning. We start by reviewing the basics of probability theory, and then discuss some approaches to probabilistic classification, including the (naive) Bayes classifier and logistic regression, which is a combination of a linear model, a sigmoid function and a log loss.
We finish with a high level overview of information theory. This gives us the concepts of entropy, cross-entropy and Kullback-Leibler Divergence, which we will make frequent use of in the lectures to come. We tie everything together by showing that log loss can be rephrased in terms of cross-entropy (which is why it often goes by the name of cross-entropy loss).










