L11.6 Xavier Glorot and Kaiming He Initialization @SebastianRaschka
L11.6 Xavier Glorot and Kaiming He Initialization  @SebastianRaschka
Uploaded March 2021 | Updated November 2026, 4 hours ago
Sebastian's books: sebastianraschka.com/books/I

MPORTANT NOTE: In the video, I talk about the number of input units in the denominator ("fan in"), but to be correct, it should have been number of input units for both the current and the next layer ("fan in" + "fan out").

Slides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L11_norm-and-init__slides.pdf

Papers:

Xavier Glorot and Yoshua Bengio. "Understanding the difficulty of training deep feedforward neural networks." Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010. http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf?hc_location=ufi]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034. 2015. arxiv.org/abs/1502.01852

-------

This video is part of my Introduction of Deep Learning course.

Next video: youtu.be/nA6oEAE9IVc

The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51

A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html

-------

If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
L11.6 Xavier Glorot and Kaiming He Initialization13.4.1 Recursive Feature Elimination (L13: Feature Selection)L14.2: Spatial Dropout and BatchNormL15.6 RNNs for Classification: A Many-to-One Word RNNBuild A Reasoning Model From Scratch 5: Inference Scaling 2 (Logprob Scoring, Self-Refinement)What I Learned From Implementing LLM Architectures From Scratch (And How to Get Started)Introduction to Generative Adversarial Networks (Tutorial Recording at ISSDL 2021)L16.1 Dimensionality ReductionL17.0 Intro to Variational Autoencoders   Lecture OverviewL5.0 Gradient Descent   Lecture Overview13.4.2 Feature Permutation Importance (L13: Feature Selection)L10.0 Regularization Methods for Neural Networks   Lecture Overview
Sebastian Raschka |

L11.6 Xavier Glorot and Kaiming He Initialization

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER