Uploaded May 2023 | Updated September 2026, 2 weeks ago

---

In this video, I show you how to accelerate Transformer training and inference with the Hugging Face Optimum Neuron library, a hardware acceleration library dedicated to AWS Trainium and AWS Inferentia 2, two custom AI chips designed by AWS.

First, changing a single line of code, I show you how to train a Vision Transformer model on the food101 datasets (75K training images). On a trn1.32xlarge instance, the model trains in under a minute per epoch.

Then, I show you how to export a DistilBERT model from the hub to Inferentia2. Running a benchmark on a inf2.xlarge instance, we get over 2000 predictions per second and P99 1-millisecond latency!

⭐️⭐️⭐️ Don't forget to subscribe to be notified of future videos ⭐️⭐️⭐️

- Amazon EC2 Trn1: aws.amazon.com/ec2/instance-types/trn1
- Amazon EC2 Inf2: aws.amazon.com/ec2/instance-types/inf2
- Hugging Face Neuron AMI: aws.amazon.com/marketplace/pp/prodview-gr3e6yiscria2
- Optimum Neuron documentation: huggingface.co/docs/optimum-neuron
- Optimum Neuron Github: github.com/huggingface/optimum-neuron
- Code: gitlab.com/juliensimon/huggingface-demos/-/tree/main/optimum/neuron