Uploaded October 2021 | Updated September 2026, 1 week ago
❤️ Become The AI Epiphany
Patreon ❤️ ► patreon.com/theaiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
In this video I cover the "Multitask Prompted Training Enables Zero-Shot Task Generalization" paper that introduced the T0 transformer.
T0 basically = Google's T5 + LM pretraining + additional training on prompted datasets.
The paper came out of the BigScience workshop.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2110.08207
✅ BigScience: bigscience.huggingface.co
✅ Models on HF hub: huggingface.co/bigscience
✅ Prompt tool: github.com/bigscience-workshop/promptsource
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 BigScience workshop, HuggingFace hub
03:00 A high-level overview
04:40 Are we really doing implicit training?
06:30 T0 training and prompt templates
10:55 Choosing the val dataset
13:20 How is T0 trained?
14:55 Results (vs GPT3)
18:25 Ablations (varying prompts and data)
21:30 Discrepancy in GPT3 API vs reported results
22:30 Outro
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany
Much love! ❤️
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar Veličković
Bartłomiej Danek
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa
Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
💻 FOLLOW ME ON GITHUB FOR ML PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#t0 #bigscience #huggingface
❤️ Become The AI Epiphany
Patreon ❤️ ► patreon.com/theaiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
In this video I cover the "Multitask Prompted Training Enables Zero-Shot Task Generalization" paper that introduced the T0 transformer.
T0 basically = Google's T5 + LM pretraining + additional training on prompted datasets.
The paper came out of the BigScience workshop.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ Paper: arxiv.org/abs/2110.08207
✅ BigScience: bigscience.huggingface.co
✅ Models on HF hub: huggingface.co/bigscience
✅ Prompt tool: github.com/bigscience-workshop/promptsource
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00 BigScience workshop, HuggingFace hub
03:00 A high-level overview
04:40 Are we really doing implicit training?
06:30 T0 training and prompt templates
10:55 Choosing the val dataset
13:20 How is T0 trained?
14:55 Results (vs GPT3)
18:25 Ablations (varying prompts and data)
21:30 Discrepancy in GPT3 API vs reported results
22:30 Outro
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany ► patreon.com/theaiepiphany
One-time donation:
paypal.com/paypalme/theaiepiphany
Much love! ❤️
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Petar Veličković
Bartłomiej Danek
Zvonimir Sabljic
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💡 The AI Epiphany is a channel dedicated to simplifying the field of AI using creative visualizations and in general, a stronger focus on geometrical and visual intuition, rather than the algebraic and numerical "intuition".
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👋 CONNECT WITH ME ON SOCIAL
LinkedIn ► linkedin.com/in/aleksagordic
Twitter ► twitter.com/gordic_aleksa
Instagram ► instagram.com/aiepiphany
Facebook ► facebook.com/aiepiphany
👨👩👧👦 JOIN OUR DISCORD COMMUNITY:
Discord ► discord.gg/peBrCpheKE
📢 SUBSCRIBE TO MY MONTHLY AI NEWSLETTER:
Substack ► aiepiphany.substack.com
💻 FOLLOW ME ON GITHUB FOR ML PROJECTS:
GitHub ► github.com/gordicaleksa
📚 FOLLOW ME ON MEDIUM:
Medium ► gordicaleksa.medium.com
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#t0 #bigscience #huggingface
