Uploaded June 2021 | Updated September 2026, 1 week ago
❤️ Become The AI Epiphany Patreon ❤️ ► patreon.com/theaiepiphany
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Paper: Do Transformers Really Perform Bad for Graph Representation?
In this video, I cover Graphormer a new transformer model that achieved SOTA results on the OGB large-scale challenge benchmark.
It achieved that by introducing 3 novel types of structural biases/encodings into the classic transformer encoder.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2106.05234
✅ What can GNNs learn blog: https://andreasloukas.blog/2019/12/27/what-gnn-can-and-cannot-learn/
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 Key points of the paper
02:15 Graph-level predictions and why we need structural information
05:35 GNNs basics
09:40 Centrality encoding explained in depth
14:15 Spatial encoding explained in depth
18:00 Edge encoding explained in depth
22:30 Results on OGB LSC
23:30 Graphormer handles over-smoothing
25:10 Other results and ablation study
28:20 Graphormer is more expressive than WL
30:30 Mean aggregation as a special case of Graphormer
35:00 Sum aggregation as a special case of Graphormer
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany
Much love! ❤️
Huge thank you to these AI Epiphany patreons:
Petar Veličković
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa
Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#graphormer #graphs #transformers
❤️ Become The AI Epiphany Patreon ❤️ ► patreon.com/theaiepiphany
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Paper: Do Transformers Really Perform Bad for Graph Representation?
In this video, I cover Graphormer a new transformer model that achieved SOTA results on the OGB large-scale challenge benchmark.
It achieved that by introducing 3 novel types of structural biases/encodings into the classic transformer encoder.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2106.05234
✅ What can GNNs learn blog: https://andreasloukas.blog/2019/12/27/what-gnn-can-and-cannot-learn/
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 Key points of the paper
02:15 Graph-level predictions and why we need structural information
05:35 GNNs basics
09:40 Centrality encoding explained in depth
14:15 Spatial encoding explained in depth
18:00 Edge encoding explained in depth
22:30 Results on OGB LSC
23:30 Graphormer handles over-smoothing
25:10 Other results and ablation study
28:20 Graphormer is more expressive than WL
30:30 Mean aggregation as a special case of Graphormer
35:00 Sum aggregation as a special case of Graphormer
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany
Much love! ❤️
Huge thank you to these AI Epiphany patreons:
Petar Veličković
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa
Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#graphormer #graphs #transformers
