Uploaded July 2023 | Updated November 2026, 3 hours ago
Let's talk about how convolution is used in NLP
Medium Article for this video: medium.com/p/573d0329cc37#e45e-15748eb5fa7d
SPONSOR
Get 20% off and be apart of a Premium Software Engineering Community for career advice and guidance: jointaro.com/r/ajayh486
ABOUT ME
β Subscribe: youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: medium.com/@dataemporium
π» Github: github.com/ajhalthor
π LinkedIn: linkedin.com/in/ajay-halthor-477974bb
PAPERS
[1 π] Phoneme detection paper with Time Delay Neural Networks: https://www.cs.toronto.edu/~fritz/absps/waibelTDNN.pdf
[2 π] TDNN rise again in 2011: jmlr.org/papers/volume12/collobert11a/collobert11a.pdf
[3 π] A Neural Probabilistic Language Model (Bengio et al., 2003): jmlr.org/papers/volume3/bengio03a/bengio03a.pdf
[4 π] Dynamic Convolution Neural Networks: arxiv.org/pdf/1404.2188.pdf
USEFUL VIDEOS
[1 π΄] Word2Vec video: youtu.be/9S0-OC4LFNo
[2 π΄] LSTM Video: youtube.com/watch?v=QciIcRxJvsM
[3 π΄] Transformer Video: youtube.com/watch?v=TQQlZhbC5ps
[4 π΄] BERT video: youtube.com/watch?v=xI0HHN5XKDo
[5 π΄] ChatGPT video: youtu.be/NpmnWgQgcsA
PLAYLISTS FROM MY CHANNEL
β Transformers from scratch playlist: youtube.com/watch?v=QCJQG4DuHT0&list=PLTl9hO2Oobd97qfWC40gOSU8C0iu0m2l4
β ChatGPT Playlist of all other videos: youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Transformer Neural Networks: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β Convolutional Neural Networks: youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : youtube.com/playlist?list=PLTl9hO2Oobd-_5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk_-h1H_W3V
β Coding Machine Learning: youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
Let's talk about how convolution is used in NLP
Medium Article for this video: medium.com/p/573d0329cc37#e45e-15748eb5fa7d
SPONSOR
Get 20% off and be apart of a Premium Software Engineering Community for career advice and guidance: jointaro.com/r/ajayh486
ABOUT ME
β Subscribe: youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: medium.com/@dataemporium
π» Github: github.com/ajhalthor
π LinkedIn: linkedin.com/in/ajay-halthor-477974bb
PAPERS
[1 π] Phoneme detection paper with Time Delay Neural Networks: https://www.cs.toronto.edu/~fritz/absps/waibelTDNN.pdf
[2 π] TDNN rise again in 2011: jmlr.org/papers/volume12/collobert11a/collobert11a.pdf
[3 π] A Neural Probabilistic Language Model (Bengio et al., 2003): jmlr.org/papers/volume3/bengio03a/bengio03a.pdf
[4 π] Dynamic Convolution Neural Networks: arxiv.org/pdf/1404.2188.pdf
USEFUL VIDEOS
[1 π΄] Word2Vec video: youtu.be/9S0-OC4LFNo
[2 π΄] LSTM Video: youtube.com/watch?v=QciIcRxJvsM
[3 π΄] Transformer Video: youtube.com/watch?v=TQQlZhbC5ps
[4 π΄] BERT video: youtube.com/watch?v=xI0HHN5XKDo
[5 π΄] ChatGPT video: youtu.be/NpmnWgQgcsA
PLAYLISTS FROM MY CHANNEL
β Transformers from scratch playlist: youtube.com/watch?v=QCJQG4DuHT0&list=PLTl9hO2Oobd97qfWC40gOSU8C0iu0m2l4
β ChatGPT Playlist of all other videos: youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Transformer Neural Networks: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β Convolutional Neural Networks: youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : youtube.com/playlist?list=PLTl9hO2Oobd-_5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk_-h1H_W3V
β Coding Machine Learning: youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
![DALL-E: Text-to-Image generation - Explained!
In this video, we take a look at a DALL-E for text-to-image generation. What is it? Why do we have it? How does it look?
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
RESOURCES
[1 π] Slides: https://link.excalidraw.com/p/readonly/NXtiUh19HjH4BuC2IQ6V
[2 π] DALL-E main paper: https://arxiv.org/pdf/2102.12092
[3 π] DALL-E blog page: https://openai.com/index/dall-e/
[4 π] Evolution of auto encoders: https://youtu.be/XyWNmHZi1oA?si=0X5iE2FKfToDaRNM
[5 π] Colab notebook I put together to understand the gumbel distribution, gumbel max trick and Gumbel Softmax Relaxation: https://colab.research.google.com/drive/1KSKB3AIUzyMnpym8HeSVZCxOtzS-DI9u#scrollTo=1af4a395
[6 π] Nice mathematical proof to show gumbel max trick: [https://github.com/priyammaz/PyTorch-Adventures/blob/main/PyTorch for Generation/AutoEncoders/Intro to AutoEncoders/gumbel_softmax_quantizer.ipynb](https://github.com/priyammaz/PyTorch-Adventures/blob/main/PyTorch%20for%20Generation/AutoEncoders/Intro%20to%20AutoEncoders/gumbel_softmax_quantizer.ipynb)
[7 π] Attention is all you need paper: https://arxiv.org/pdf/1706.03762
[8 π] Image is worth 16 x 16 words paper: https://arxiv.org/pdf/2010.11929
[9 π] Improving generative language understanding paper: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
[10 π] Learning Bounded Context-Free-Grammar via LSTM and the Transformer:
Difference and Explanations paper: https://arxiv.org/pdf/2112.09174
[11 π] DALL-E architecture code: https://github.com/openai/DALL-E/blob/master/dall_e/encoder.py
PLAYLISTS FROM MY CHANNEL
β Reinforcement Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β Transformers from Scratch: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Convolutional Neural Networks: https://youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : https://youtube.com/playlist?list=PLTl9hO2Oobd 5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk h1H_W3V
β Coding Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow
CHAPTERS
00:00 What is DALL-E?
00:33 Why DALL-E with historical context
03:35 Components of DALL-E: dVAE and GPT
04:39 Stage 1: discrete VAE training
08:00 Stage 2: GPT training
11:38 Inference
13:36 dVAE encoder
15:58 dVAE image tokenizer
17:33 dVAE decoder
18:14 dVAE loss
20:56 Gumbel Distribution
23:20 Gumbel Max Trick
27:27 Gumbel Softmax Relaxation
29:20 Quiz Time
30:17 Summary DALL-E: Text-to-Image generation - Explained!](https://i.ytimg.com/vi/Al-FrFri4pI/mqdefault.jpg)
![Informer attention code - FROM SCRATCH!
In this video, we code the prob sparse attention and compare it to time series attention
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
RESOURCES
[1] Main repo: https://github.com/zhouhaoyi/Informer2020/blob/main/models/attn.py
[2] Code for the colab notebook: https://github.com/ajhalthor/Informer/tree/main
PLAYLISTS FROM MY CHANNEL
β Deep Learning 101: https://www.youtube.com/playlist?list=PLTl9hO2Oobd_NwyY_PeSYrYfsvHZnHGPU
β Natural Language Processing 101: https://www.youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β Reinforcement Learning 101: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
β Transformers from Scratch: https://www.youtube.com/playlist?list=PLTl9hO2Oobd97qfWC40gOSU8C0iu0m2l4
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow Informer attention code - FROM SCRATCH!](https://i.ytimg.com/vi/B0phjAyaY3Q/mqdefault.jpg)


![Where did convolution networks come from?
Lets take a look at how convolution neural networks evolved diving into an early network called Neo-cognitron. Hopefully, we can answer questions like:
1. Why do we use convolution, activation, pooling?
2. Why do the networks have layered architectures
3. What exactly is being learned in a convolution network
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
RESOURCES
[1 π] Slides: https://link.excalidraw.com/p/readonly/o0yE3HWfPMaBcu9QBPNf
[2 π] Main paper for neocognitron: https://www.rctn.org/bruno/public/papers/Fukushima1980.pdf
[3 π] Great demo (start at 3:12) : https://youtu.be/KAazjZoiCd0?si=WPNUg4YKBbJtlwsD
[4 π] Original Paper from Hubel and Wiesels experiment: https://pmc.ncbi.nlm.nih.gov/articles/PMC1363130/pdf/jphysiol01298-0128.pdf
PLAYLISTS FROM MY CHANNEL
β Reinforcement Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β Transformers from Scratch: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Convolutional Neural Networks: https://youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : https://youtube.com/playlist?list=PLTl9hO2Oobd 5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk h1H_W3V
β Coding Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow
CHAPTERS
00:00 Hubel & Weiselβs Experiment
02:00 Precoursor to CNNs
03:40 Why layered organization?
04:50 Simple Cells in biology
06:40 Simple Cells in computation model (relu activation)
08:56 Primary Visual Cortex in biology
10:28 Primary Visual Cortex in computation model
11:58 Where does βconvolutionβ come in?
15:38 Complex Cells in biology
16:57 Complex Cells in computation model (pooling)
18:04 convolution + activation + pooling
21:15 How does the network learn?
30:22 Conclusion
30:46 Quiz Time
31:40 Summary Where did convolution networks come from?](https://i.ytimg.com/vi/BClejohiShQ/mqdefault.jpg)
![Knowledge Distillation in Neural Networks - Explained!
In this video, we take a look at Knowledge Distillation. What is it? Why do we have it? How does it work?
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
RESOURCES
[1 π] Slides: https://link.excalidraw.com/p/readonly/rBinJxKL9ogituDfxqJn
[2 π] 2006 paper that introduced Model Compression: https://www.cs.cornell.edu/~caruana/compression.kdd06.pdf?utm_source=chatgpt.com
[3 π] 2014 paper that transfers dark knowledge: https://arxiv.org/pdf/1312.6184
[4 π] 2015 paper on knowledge distillation (main paper): https://arxiv.org/pdf/1503.02531
PLAYLISTS FROM MY CHANNEL
β Reinforcement Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β Transformers from Scratch: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Convolutional Neural Networks: https://youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : https://youtube.com/playlist?list=PLTl9hO2Oobd 5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk h1H_W3V
β Coding Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow
CHAPTERS
00:00 What is Knowledge Distillation?
00:26 Why Knowledge Distillation
01:52 Model Compression
03:10 Logit Matching
04:17 Combining the two with Knowledge distillation
05:49 How is Knowledge Distillation is done?
09:47 Knowledge Distillation vs Logit Matching
10:27 Quiz Time
11:27 Summary Knowledge Distillation in Neural Networks - Explained!](https://i.ytimg.com/vi/BUCSTKQOzcM/mqdefault.jpg)
![Why do CNNs look like THIS?
How and why do convolution networks look the way they do? Lets talk about that
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
RESOURCES
[1 π] LeNet-1 (good paper to get intuition on how CNNs are constructed): https://masters.donntu.ru/2012/fknt/umiarov/library/lecun.pdf
[2 π] LeNet-5 paper: http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf
[3 π] AlexNet paper: https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
[4 π] ResNet paper: https://arxiv.org/pdf/1512.03385
[5 π] U-Net paper: https://arxiv.org/pdf/1505.04597
[6 π] YOLO paper: https://arxiv.org/pdf/1506.02640
[7 π] Transformer Paper: https://arxiv.org/pdf/1706.03762
PLAYLISTS FROM MY CHANNEL
β Reinforcement Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β Transformers from Scratch: https://www.youtube.com/playlist?list=PLTl9hO2Oobd97qfWC40gOSU8C0iu0m2l4
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Convolutional Neural Networks: https://youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : https://youtube.com/playlist?list=PLTl9hO2Oobd 5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk h1H_W3V
β Coding Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow Why do CNNs look like THIS?](https://i.ytimg.com/vi/Bl31dnuiH9I/mqdefault.jpg)
![Why Do Transformers Look Like This?
Why do Transformers look like this? There is a story behind that.
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
RESOURCES
[1 π] Transformer Paper: https://arxiv.org/pdf/1706.03762
[2 π] LSTM blog: https://karpathy.github.io/2015/05/21/rnn-effectiveness/
[3 π] Transformer Architecture Image deep dive: https://github.com/ajhalthor/Transformer-Neural-Network/blob/main/Transformer_Architecture_complete.png
PLAYLISTS FROM MY CHANNEL
β Reinforcement Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β Transformers from Scratch: https://www.youtube.com/playlist?list=PLTl9hO2Oobd97qfWC40gOSU8C0iu0m2l4
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
β Convolutional Neural Networks: https://youtube.com/playlist?list=PLTl9hO2Oobd9U0XHz62Lw6EgIMkQpfz74
β The Math You Should Know : https://youtube.com/playlist?list=PLTl9hO2Oobd 5sGLnbgE8Poer1Xjzz4h
β Probability Theory for Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd9bPcq0fj91Jgk h1H_W3V
β Coding Machine Learning: https://youtube.com/playlist?list=PLTl9hO2Oobd82vcsOnvCNzxrZOlrz3RiD
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow Why Do Transformers Look Like This?](https://i.ytimg.com/vi/BnjGGiEIXeQ/mqdefault.jpg)
![LoRA - Explained!
A parameter efficient fine tuning technique that makes use of a low rank adapter to (1) reduce storage required per task by decreasing the number of trainable parameters added to the network per task (2) remove inference latency ensuring the stored parameters are applied to the existing network architecture instead of adding more
RESOURCES
[1 π] Paper that introduces LoRA: https://arxiv.org/pdf/2106.09685
[2 π] Paper that introduced the additive adapter: https://arxiv.org/abs/2312.12148
[3 π] Other PEFT techniques: https://arxiv.org/pdf/2312.12148
[4 π] My video on Parameter Efficient Fine Tuning (PEFT): https://youtu.be/HcVtpLAGMXo
[5 π] Neural networks tend to be more complex than required, hence we can make use of a low rank adapter without losing performance accuracy: https://arxiv.org/pdf/1804.08838
ABOUT ME
β Subscribe: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1
π Medium Blog: https://medium.com/@dataemporium
π» Github: https://github.com/ajhalthor
π LinkedIn: https://www.linkedin.com/in/ajay-halthor-477974bb/
PLAYLISTS FROM MY CHANNEL
β Deep Learning 101: https://www.youtube.com/playlist?list=PLTl9hO2Oobd_NwyY_PeSYrYfsvHZnHGPU
β Natural Language Processing 101: https://www.youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β Reinforcement Learning 101: https://youtube.com/playlist?list=PLTl9hO2Oobd9kS NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing 101: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β Transformers from Scratch: https://youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β ChatGPT Playlist: https://youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ
MATH COURSES (7 day free trial)
π Mathematics for Machine Learning: https://imp.i384100.net/MathML
π Calculus: https://imp.i384100.net/Calculus
π Statistics for Data Science: https://imp.i384100.net/AdvancedStatistics
π Bayesian Statistics: https://imp.i384100.net/BayesianStatistics
π Linear Algebra: https://imp.i384100.net/LinearAlgebra
π Probability: https://imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
π β Deep Learning Specialization: https://imp.i384100.net/Deep-Learning
π Python for Everybody: https://imp.i384100.net/python
π MLOps Course: https://imp.i384100.net/MLOps
π Natural Language Processing (NLP): https://imp.i384100.net/NLP
π Machine Learning in Production: https://imp.i384100.net/MLProduction
π Data Science Specialization: https://imp.i384100.net/DataScience
π Tensorflow: https://imp.i384100.net/Tensorflow
CHAPTERS
0:00 Introduction
1:49 Pass 1: Low Rank Matrices
8:00 Quiz 1
8:52 Pass 2: Adapters
16:38 Quiz 2
17:47 Pass 3: Low Rank Adapters
26:37 Quiz 3
27:54 Summary LoRA - Explained!](https://i.ytimg.com/vi/Bq9zqTJDsjg/mqdefault.jpg)

