LLM (Parameter Efficient) Fine Tuning - Explained! @CodeEmporium
LLM (Parameter Efficient) Fine Tuning - Explained!  @CodeEmporium
Uploaded September 2024 | Updated November 2026, 2 hours ago
Parameter efficient fine tuning is increasingly important in NLP and genAI. Let's talk about it.

RESOURCES
[1 πŸ“š] RNNs were the SOTA for sequence tasks: arxiv.org/pdf/1409.0473
[2 πŸ“š] Then transformers came on the scene: arxiv.org/pdf/1706.03762
[3 πŸ“š] Pretraining and Finetuning architectures like BERT came along: arxiv.org/pdf/1810.04805
[4 πŸ“š] But LLMs are huge: informationisbeautiful.net/visualizations/the-rise-of-generative-ai-large-language-models-llms-like-chatgpt
[5 πŸ“š] Few shot learning by GPT-3 tries to address the issue: arxiv.org/pdf/2005.14165
[6 πŸ“š] Parameter Efficient Transfer Learning reduces the trainable parameters via additive adapters (the first PEFT technique): arxiv.org/pdf/1902.00751
[7 πŸ“š] Since 2019, there have been many PEFT techniques introduced: arxiv.org/pdf/2312.12148
[8 πŸ“š] Other notable techniques include prefix-tuning: arxiv.org/pdf/2101.00190
[9 πŸ“š] And LoRA: arxiv.org/pdf/2106.09685
[10 πŸ“š] And a quantized version of LoRA called QLoRA: arxiv.org/pdf/2305.14314
[11 πŸ“š] We see these adapters in use in LLMs today like Llama: arxiv.org/pdf/2303.16199


ABOUT ME
β­• Subscribe: youtube.com/c/CodeEmporium?sub_confirmation=1
πŸ“š Medium Blog: medium.com/@dataemporium
πŸ’» Github: github.com/ajhalthor
πŸ‘” LinkedIn: linkedin.com/in/ajay-halthor-477974bb


PLAYLISTS FROM MY CHANNEL
β­• Deep Learning 101: youtube.com/playlist?list=PLTl9hO2Oobd_NwyY_PeSYrYfsvHZnHGPU
β­• Natural Language Processing 101: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β­• Reinforcement Learning 101: youtube.com/playlist?list=PLTl9hO2Oobd9kS--NgVz0EPNyEmygV1Ha&si=AuThDZJwG19cgTA8
Natural Language Processing 101: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE&si=LsVy8RDPu8jeO-cc
β­• Transformers from Scratch: youtube.com/playlist?list=PLTl9hO2Oobd_bzXUpzKMKA3liq2kj6LfE
β­• ChatGPT Playlist: youtube.com/playlist?list=PLTl9hO2Oobd9coYT6XsTraTBo4pL1j4HJ

CHAPTERS
0:00 Introduction
1:00 Pass 1: What & Why PEFT
6:27 Quiz 1
7:26 Pass 2: Details
16:20 Quiz 2
17:11 Pass 3: Performance Evaluation
20:49 Quiz 3
21:43 Summary

MATH COURSES (7 day free trial)
πŸ“• Mathematics for Machine Learning: imp.i384100.net/MathML
πŸ“• Calculus: imp.i384100.net/Calculus
πŸ“• Statistics for Data Science: imp.i384100.net/AdvancedStatistics
πŸ“• Bayesian Statistics: imp.i384100.net/BayesianStatistics
πŸ“• Linear Algebra: imp.i384100.net/LinearAlgebra
πŸ“• Probability: imp.i384100.net/Probability

OTHER RELATED COURSES (7 day free trial)
πŸ“• ⭐ Deep Learning Specialization: imp.i384100.net/Deep-Learning
πŸ“• Python for Everybody: imp.i384100.net/python
πŸ“• MLOps Course: imp.i384100.net/MLOps
πŸ“• Natural Language Processing (NLP): imp.i384100.net/NLP
πŸ“• Machine Learning in Production: imp.i384100.net/MLProduction
πŸ“• Data Science Specialization: imp.i384100.net/DataScience
πŸ“• Tensorflow: imp.i384100.net/Tensorflow
LLM (Parameter Efficient) Fine Tuning - Explained!Vision Transformers - Explained!5 concepts in transformer neural networks (Part 1)What is a policy in Reinforcement Learning? #reinforcementlearning #artificialintelligence e youInception Net - Explained! (with code)BERT Pretaining summary #deeplearning  #machinelearningEncoder Decoder Architectures for RNNs #deeplearning #machinelearningSwin transformer - Explained!What are RNNs?Embeddings - EXPLAINED!Optimizers in Neural Networks - EXPLAINED!Vector to Sequence Recurrent Neural Network #deeplearning #machinelearning
CodeEmporium |

LLM (Parameter Efficient) Fine Tuning - Explained!

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER