DINO - Explained! @CodeEmporium
DINO - Explained!  @CodeEmporium
Uploaded March 2026 | Updated November 2026, 3 hours ago
In this video, we take a look at DIstillation with NO labels. What is it? Why do we have it? How does it look?

ABOUT ME
⭕ Subscribe: youtube.com/c/CodeEmporium?sub_confirmation=1
📚 Medium Blog: medium.com/@dataemporium
💻 Github: github.com/ajhalthor
👔 LinkedIn: linkedin.com/in/ajay-halthor-477974bb

RESOURCES
[1 📚] Main Paper: arxiv.org/pdf/2104.14294
[2 📚] Slides: link.excalidraw.com/p/readonly/ccVu9FUIwD5miDWgdK3s
[3 📚] Vision Transformers paper: arxiv.org/pdf/2010.11929
[4 📚] BERT paper: arxiv.org/pdf/1810.04805

PLAYLISTS FROM MY CHANNEL
⭕ Reinforcement Learning: youtube.com/playlist?list=PLTl9hO2Oobd9kS--NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
⭕ Transformers from Scratch: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
⭕ ChatGPT Playlist: youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
⭕ Convolutional Neural Networks: youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
⭕ The Math You Should Know : youtube.com/playlist?list=PLTl9hO2Oobd-_5sGLnbgE8Poer1Xjzz4h
⭕ Probability Theory for Machine Learning: youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk_-h1H_W3V
⭕ Coding Machine Learning: youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD


MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: imp.i384100.net/MathML
📕 Calculus: imp.i384100.net/Calculus
📕 Statistics for Data Science: imp.i384100.net/AdvancedStatistics
📕 Bayesian Statistics: imp.i384100.net/BayesianStatistics
📕 Linear Algebra: imp.i384100.net/LinearAlgebra
📕 Probability: imp.i384100.net/Probability

OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: imp.i384100.net/Deep-Learning
📕 Python for Everybody: imp.i384100.net/python
📕 MLOps Course: imp.i384100.net/MLOps
📕 Natural Language Processing (NLP): imp.i384100.net/NLP
📕 Machine Learning in Production: imp.i384100.net/MLProduction
📕 Data Science Specialization: imp.i384100.net/DataScience
📕 Tensorflow: imp.i384100.net/Tensorflow

CHAPTERS
00:00 What is DINO?
00:24 Historical context: Vision Transformers Recap
02:40 Self supervised learning
04:51 Student-teacher architecture as we do in knowledge distillation
05:12 Training DINO: forward pass
09:43 Why is the cardinality of output neurons large?
10:27 temperature softmax in the teacher and student
11:43 mode collapse and reason for centering teacher activations
13:10 How the student and teacher update weights
15:22 Inference
17:50 Interesting Findings
18:30 visualizing segmentation masks that emerge in ViT
20:59 understanding rich image embeddings of ViT
22:06 Quiz Time
22:57 Summary
DINO - Explained!Building a Translator with TransformersLimits and inifinity in calculus10 Papers that changed Computer VisionDid Transformers KILL CNNs?Diffusion Models - Explained!History of AI in TranslationWhat network uses convolution?Why neural networks are so deep? (AlexNet - Explained)What are Foundation Models?Forgotten history of Calculus in IndiaSequence to Vector Recurrent Neural Networks  #deeplearning #machinelearning
CodeEmporium |

DINO - Explained!

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER