Uploaded September 2025 | Updated September 2026, 3 weeks ago
Ever wondered how large language models like GPT are actually trained? In this video, I break down the full pre-training process step by step, from tokenization and context windows to batches, loss functions, backpropagation, and weight updates.
For deeper dives, I highly recommend the excellent Under The Hood videos on transformers:
youtube.com/watch?v=FtFZ-vBA-eA&t=1003s
Twitter: twitter.com/Hashlipsnft
Website: hashlips.io
HashLips/HashLips Lab provides educational content and open-source code for informational purposes only, without any express or implied warranties on accuracy or completeness. Our materials are not intended as financial or professional advice and should not replace professional judgment or expertise. The use of our content and code is at your sole discretion and risk.
Please note, that our resources are not financial advice. Decisions made based on our content are the user's responsibility, and HashLips/HashLips Lab assumes no liability for any direct or indirect losses, including but not limited to data loss or profit loss, that may result from utilizing our educational materials or open-source code.
#ai #llms
Ever wondered how large language models like GPT are actually trained? In this video, I break down the full pre-training process step by step, from tokenization and context windows to batches, loss functions, backpropagation, and weight updates.
For deeper dives, I highly recommend the excellent Under The Hood videos on transformers:
youtube.com/watch?v=FtFZ-vBA-eA&t=1003s
Twitter: twitter.com/Hashlipsnft
Website: hashlips.io
HashLips/HashLips Lab provides educational content and open-source code for informational purposes only, without any express or implied warranties on accuracy or completeness. Our materials are not intended as financial or professional advice and should not replace professional judgment or expertise. The use of our content and code is at your sole discretion and risk.
Please note, that our resources are not financial advice. Decisions made based on our content are the user's responsibility, and HashLips/HashLips Lab assumes no liability for any direct or indirect losses, including but not limited to data loss or profit loss, that may result from utilizing our educational materials or open-source code.
#ai #llms
