Uploaded July 2024 | Updated September 2026, 1 week ago
In this StatQuest we walk through the code required to code your own ChatGPT like Transformer in PyTorch and we do it one step at a time, with every little detail clearly explained.
NOTE: This StatQuest assumes that you are already familiar with the concepts behind...
Decoder-Only Transformers: youtu.be/bQ5BoolX9Ag
The Essential Matrix Algebra for Neural Networks: youtu.be/ZTt9gsGcdDo
The Matrix Math Behind Transformers: youtu.be/KphmOJnLAdI
You can get the code here: github.com/StatQuest/decoder_transformer_from_scratch
The full Neural Networks playlist, from the basics to AI, is here: youtube.com/watch?v=CqOfi41LfDw&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1
Learn more about GiveInternet.org: giveinternet.org/StatQuest NOTE: Donations up to $30 will be matched by an Angel Investor - so a $30 donation would give $60 to the organization. DOUBLE BAM!!!
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index
If you'd like to support StatQuest, please consider...
Patreon: patreon.com/statquest
...or...
YouTube Membership: youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join
...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
statquest.org/statquest-store
...or just donating to StatQuest!
paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
twitter.com/joshuastarmer
0:00 Awesome song and introduction
1:12 Loading the modules
2:04 Creating the training dataset
6:17 Coding Position Encoding
14:09 Coding Attention
21:04 Coding a Decoder-Only Transformer
26:39 Running the model (untrained)
29:18 Training and using the model
#StatQuest #PyTorch #chatgpt
In this StatQuest we walk through the code required to code your own ChatGPT like Transformer in PyTorch and we do it one step at a time, with every little detail clearly explained.
NOTE: This StatQuest assumes that you are already familiar with the concepts behind...
Decoder-Only Transformers: youtu.be/bQ5BoolX9Ag
The Essential Matrix Algebra for Neural Networks: youtu.be/ZTt9gsGcdDo
The Matrix Math Behind Transformers: youtu.be/KphmOJnLAdI
You can get the code here: github.com/StatQuest/decoder_transformer_from_scratch
The full Neural Networks playlist, from the basics to AI, is here: youtube.com/watch?v=CqOfi41LfDw&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1
Learn more about GiveInternet.org: giveinternet.org/StatQuest NOTE: Donations up to $30 will be matched by an Angel Investor - so a $30 donation would give $60 to the organization. DOUBLE BAM!!!
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index
If you'd like to support StatQuest, please consider...
Patreon: patreon.com/statquest
...or...
YouTube Membership: youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join
...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
statquest.org/statquest-store
...or just donating to StatQuest!
paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
twitter.com/joshuastarmer
0:00 Awesome song and introduction
1:12 Loading the modules
2:04 Creating the training dataset
6:17 Coding Position Encoding
14:09 Coding Attention
21:04 Coding a Decoder-Only Transformer
26:39 Running the model (untrained)
29:18 Training and using the model
#StatQuest #PyTorch #chatgpt
