Uploaded June 2022 | Updated September 2026, 1 week ago
β€οΈ Become The AI Epiphany Patreon β€οΈ
patreon.com/theaiepiphany
π¨βπ©βπ§βπ¦ Join our Discord community π¨βπ©βπ§βπ¦
discord.gg/peBrCpheKE
Kicking off a series of videos where I'll be going through the actual code of many of the papers I've covered over the last few years!
In this video I do a code walkthrough of OpenAI's CLIP model from the "Learning Transferable Visual Models From Natural Language Supervision" paper.
Let me know what you'd like me to cover next!
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
β GitHub: github.com/openai/CLIP
β Paper: arxiv.org/abs/2103.00020
Learn about Byte-Pair Encoding:
β en.wikipedia.org/wiki/Byte_pair_encoding
β leimao.github.io/blog/Byte-Pair-Encoding
β Video: youtube.com/watch?v=tOMjTCO0htA&ab_channel=FromLanguagestoInformation
β youtube.com/watch?v=zjaRNfvNMTs&ab_channel=AbhishekThakur
Unicode:
β dmitripavlutin.com/what-every-javascript-developer-should-know-about-unicode
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
βοΈ Timetable:
00:00:00 Intro
00:02:00 High level overview: Interacting with CLIP
00:26:11 High level overview: Prompt engineering for ImageNet
00:40:25 Deep dive starts: vocabulary and byte-pair encoding
00:49:00 Vision Transformer & Text Transformer explained
01:02:00 Tokenization walkthrough
01:09:25 Encoding the image
01:15:15 Encoding the text
01:23:15 Learning a linear probe
01:27:00 Tokenization of the (brain emoji)
01:29:56 Outro
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
π° BECOME A PATREON OF THE AI EPIPHANY β€οΈ
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany - patreon.com/theaiepiphany
One-time donation - paypal.com/paypalme/theaiepiphany
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Kevin Stone
Petar VeliΔkoviΔ
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
πΌ LinkedIn - linkedin.com/in/aleksagordic
π¦ Twitter - twitter.com/gordic_aleksa
π¨βπ©βπ§βπ¦ Discord - discord.gg/peBrCpheKE
πΊ YouTube - youtube.com/c/TheAIEpiphany
π Medium - gordicaleksa.medium.com
π» GitHub - github.com/gordicaleksa
π’ AI Newsletter - aiepiphany.substack.com
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
#CLIP #contrastive #codewalkthrough
β€οΈ Become The AI Epiphany Patreon β€οΈ
patreon.com/theaiepiphany
π¨βπ©βπ§βπ¦ Join our Discord community π¨βπ©βπ§βπ¦
discord.gg/peBrCpheKE
Kicking off a series of videos where I'll be going through the actual code of many of the papers I've covered over the last few years!
In this video I do a code walkthrough of OpenAI's CLIP model from the "Learning Transferable Visual Models From Natural Language Supervision" paper.
Let me know what you'd like me to cover next!
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
β GitHub: github.com/openai/CLIP
β Paper: arxiv.org/abs/2103.00020
Learn about Byte-Pair Encoding:
β en.wikipedia.org/wiki/Byte_pair_encoding
β leimao.github.io/blog/Byte-Pair-Encoding
β Video: youtube.com/watch?v=tOMjTCO0htA&ab_channel=FromLanguagestoInformation
β youtube.com/watch?v=zjaRNfvNMTs&ab_channel=AbhishekThakur
Unicode:
β dmitripavlutin.com/what-every-javascript-developer-should-know-about-unicode
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
βοΈ Timetable:
00:00:00 Intro
00:02:00 High level overview: Interacting with CLIP
00:26:11 High level overview: Prompt engineering for ImageNet
00:40:25 Deep dive starts: vocabulary and byte-pair encoding
00:49:00 Vision Transformer & Text Transformer explained
01:02:00 Tokenization walkthrough
01:09:25 Encoding the image
01:15:15 Encoding the text
01:23:15 Learning a linear probe
01:27:00 Tokenization of the (brain emoji)
01:29:56 Outro
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
π° BECOME A PATREON OF THE AI EPIPHANY β€οΈ
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany - patreon.com/theaiepiphany
One-time donation - paypal.com/paypalme/theaiepiphany
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Kevin Stone
Petar VeliΔkoviΔ
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
πΌ LinkedIn - linkedin.com/in/aleksagordic
π¦ Twitter - twitter.com/gordic_aleksa
π¨βπ©βπ§βπ¦ Discord - discord.gg/peBrCpheKE
πΊ YouTube - youtube.com/c/TheAIEpiphany
π Medium - gordicaleksa.medium.com
π» GitHub - github.com/gordicaleksa
π’ AI Newsletter - aiepiphany.substack.com
β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬β¬
#CLIP #contrastive #codewalkthrough
