Uploaded June 2021 | Updated September 2026, 1 week ago

---

❤️ Become The AI Epiphany Patreon ❤️ ► patreon.com/theaiepiphany
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

When Vision Transformers Outperform ResNets without Pretraining or Strong Data Augmentation paper explained.

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2106.01548
✅ LinkedIn post: linkedin.com/posts/aleksagordic_vision-transformers-mlp-activity-6807372257187442688-7jzF
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

⌚️ Timetable:
00:00 Key points of the paper
01:37 Key conclusions
03:00 Inductive biases and biases in a CNN
07:00 SAM explained
11:30 Possibility of heavy pruning, overfitting, sparsity, etc.
14:20 Neural tangent kernel and steepness of curvature
17:30 Results, empirical correlation between SAM and biases
19:00 Deeper look into the Hessians
20:50 Attention visualized, low data regime plots

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️

If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!

The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany

Much love! ❤️

Huge thank you to these AI Epiphany patreons:
Petar Veličković
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa

Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany

👨‍👩‍👧‍👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE

📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com

💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa

📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

#visiontransformer #nopretraining #resnets