Uploaded April 2021 | Updated September 2026, 1 day ago
In this first part video we talk about how Google Translate probably works, and a little bit of some general theory behind Neural Machine Translation (NMT). Specifically, we touch on transformers, a common NLP model for sequential data, and how they rely on a specific type of attention called self-attention. We load up our pretrained model, mt5, using the Hugging Face transformer library and model repository.
Relevant Links:
GitHub repo: github.com/ejmejm/multilingual-nmt-mt5
Colab code: colab.research.google.com/drive/1eGSCod03SjWD_YOfwb33kMJOeDZGu7lP?usp=sharing
Transformer paper: arxiv.org/abs/1706.03762
mT5 paper: arxiv.org/abs/2010.11934
Article on how transformers work: towardsdatascience.com/transformers-141e32e69591
In this first part video we talk about how Google Translate probably works, and a little bit of some general theory behind Neural Machine Translation (NMT). Specifically, we touch on transformers, a common NLP model for sequential data, and how they rely on a specific type of attention called self-attention. We load up our pretrained model, mt5, using the Hugging Face transformer library and model repository.
Relevant Links:
GitHub repo: github.com/ejmejm/multilingual-nmt-mt5
Colab code: colab.research.google.com/drive/1eGSCod03SjWD_YOfwb33kMJOeDZGu7lP?usp=sharing
Transformer paper: arxiv.org/abs/1706.03762
mT5 paper: arxiv.org/abs/2010.11934
Article on how transformers work: towardsdatascience.com/transformers-141e32e69591
