Uploaded January 2024 | Updated September 2026, 2 weeks ago

---

Explore the technical intricacies of optimizing Hugging Face models on AWS accelerators in this detailed walkthrough, possibly the most complete and most up-to-date available today.

⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos. Follow me on Medium at julsimon.medium.com or Substack at https://julsimon.substack.com. ⭐️⭐️⭐️

This video focuses on the hardware and software details essential for achieving peak performance. Access relevant code snippets and developer resources, suitable for both newcomers and experienced professionals. Whether you're familiar with Trainium and Inferentia2 or approaching these technologies for the first time, this technical walkthrough ensures your readiness for success in deploying Hugging Face models on AWS.

Dive into all key components!

00:00 Introduction
05:00 AWS NeuronCore-v2
10:30 AWS Trainium
13:48 AWS Inferentia2
16:25 Amazon EC2 Trn1
20:12 Amazon EC2 Inf2
23:20 AWS Neuron SDK
30:00 AWS Neuronx Distributed
35:25 AWS Transformers Neuronx
41:41 Hugging Face Optimum Neuron training and inference

Links:
- AWS Neuron SDK: awsdocs-neuron.readthedocs-hosted.com
- Hugging Face Optimum Neuron: huggingface.co/docs/optimum-neuron/index