Uploaded March 2019 | Updated September 2026, 3 days ago
ERRATA: In slide 55, W_i and W_c are the wrong way around in the definition of a_t.
slides: mlvu.github.io/lectures/52.Trees.annotated.pdf
course materials: mlvu.github.io
In this lecture we (finally) show how decision trees are trained. We discuss the basic tree learning algorithm (sometimes called ID3 or C45) and how to extend the principle to decision trees with numeric features and decision tree with numeric outputs (also known as regression trees).
We also dicuss ensembling: a technique that can be used to combine collections of machine learning models into a more powerful single model. We discuss bagging, boosting and stacking.
For boosting, we go into some detail for two variants: AdaBoost, and Gradient boosting. Gradient Boosted regression trees are probably one of the most popular models in the classical machine learning setting.
ERRATA: In slide 55, W_i and W_c are the wrong way around in the definition of a_t.
slides: mlvu.github.io/lectures/52.Trees.annotated.pdf
course materials: mlvu.github.io
In this lecture we (finally) show how decision trees are trained. We discuss the basic tree learning algorithm (sometimes called ID3 or C45) and how to extend the principle to decision trees with numeric features and decision tree with numeric outputs (also known as regression trees).
We also dicuss ensembling: a technique that can be used to combine collections of machine learning models into a more powerful single model. We discuss bagging, boosting and stacking.
For boosting, we go into some detail for two variants: AdaBoost, and Gradient boosting. Gradient Boosted regression trees are probably one of the most popular models in the classical machine learning setting.
