Uploaded May 2021 | Updated September 2026, 2 weeks ago
Sebastian's books: sebastianraschka.com/books
Slides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf
00:00 Introducing self attention and transformer networks.
02:05 Introduction to RNNs with an Attention Mechanism
04:08 Attention Mechanism is a foundational concept in transformer architecture.
06:07 Introduction to self attention mechanism in transformers
08:04 RNNs with Attention Mechanism use weighted sum to compute attention value
10:32 RNNs with Attention Mechanism involve computing normalized attention weights using softmax function.
12:24 RNNs with attention use dot product to compute similarity.
14:29 Word embeddings in RNNs provide consistent values regardless of word position.
Crafted by Merlin AI.
-------
This video is part of my Introduction of Deep Learning course.
Next video: youtu.be/0PjHri8tc1c
The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51
A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html
-------
If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
Sebastian's books: sebastianraschka.com/books
Slides: sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf
00:00 Introducing self attention and transformer networks.
02:05 Introduction to RNNs with an Attention Mechanism
04:08 Attention Mechanism is a foundational concept in transformer architecture.
06:07 Introduction to self attention mechanism in transformers
08:04 RNNs with Attention Mechanism use weighted sum to compute attention value
10:32 RNNs with Attention Mechanism involve computing normalized attention weights using softmax function.
12:24 RNNs with attention use dot product to compute similarity.
14:29 Word embeddings in RNNs provide consistent values regardless of word position.
Crafted by Merlin AI.
-------
This video is part of my Introduction of Deep Learning course.
Next video: youtu.be/0PjHri8tc1c
The complete playlist: youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51
A handy overview page with links to the materials: sebastianraschka.com/blog/2021/dl-course.html
-------
If you want to be notified about future videos, please consider subscribing to my channel: youtube.com/c/SebastianRaschka
