Uploaded July 2021 | Updated September 2026, 1 week ago
❤️ Become The AI Epiphany Patreon ❤️ ► patreon.com/theaiepiphany
In this video I cover VQ-GAN or Taming Transformers for High-Resolution Image Synthesis.
It uses modified VQ-VAEs and a powerful transformer (GPT-2) to synthesize high-res images.
An important modification of VQ-VAE they brought are:
1) changing MSE for perceptual loss
2) adding adversarial loss which makes the images way more crispy compared to the original VQ-VAE which had blurry outputs.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2012.09841
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 Intro
01:50 A high-level VQ-GAN overview
04:00 Perceptual loss
05:10 Patch-based adversarial loss
06:45 Sequence prediction via GPT
09:50 Generating high-res images
12:45 Loss explained in depth
16:15 Training the transformer
17:50 Conditioning transformer
20:45 Comparisons and results
22:00 Sampling strategies
23:00 Comparisons and results continued
25:00 Rejection sampling with ResNet or CLIP
26:45 Receptive field effects
28:30 Comparisons with DALL-E
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany
Much love! ❤️
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar Veličković
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa
Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#vqvae #imagesynthesis #gpt
❤️ Become The AI Epiphany Patreon ❤️ ► patreon.com/theaiepiphany
In this video I cover VQ-GAN or Taming Transformers for High-Resolution Image Synthesis.
It uses modified VQ-VAEs and a powerful transformer (GPT-2) to synthesize high-res images.
An important modification of VQ-VAE they brought are:
1) changing MSE for perceptual loss
2) adding adversarial loss which makes the images way more crispy compared to the original VQ-VAE which had blurry outputs.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2012.09841
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 Intro
01:50 A high-level VQ-GAN overview
04:00 Perceptual loss
05:10 Patch-based adversarial loss
06:45 Sequence prediction via GPT
09:50 Generating high-res images
12:45 Loss explained in depth
16:15 Training the transformer
17:50 Conditioning transformer
20:45 Comparisons and results
22:00 Sampling strategies
23:00 Comparisons and results continued
25:00 Rejection sampling with ResNet or CLIP
26:45 Receptive field effects
28:30 Comparisons with DALL-E
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany
Much love! ❤️
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar Veličković
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa
Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
💻 FOLLOW ME ON GITHUB FOR COOL PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#vqvae #imagesynthesis #gpt
