75 years of Language Models @CodeEmporium
75 years of Language Models  @CodeEmporium
Uploaded August 2023 | Updated November 2026, 3 hours ago
Here are 20 research papers you should read to master Language modeling.

ABOUT ME
โญ• Subscribe: youtube.com/c/CodeEmporium?sub_confirmation=1
๐Ÿ“š Medium Blog: medium.com/@dataemporium
๐Ÿ’ป Github: github.com/ajhalthor
๐Ÿ‘” LinkedIn: linkedin.com/in/ajay-halthor-477974bb

PAPERS
[1 ๐Ÿ“š] A Mathematical Theory of Communication: https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf
[2 ๐Ÿ“š] A Neural Probabilistic Language Model: jmlr.org/papers/volume3/bengio03a/bengio03a.pdf
[3 ๐Ÿ“š] NLP (Almost) from scratch: jmlr.org/papers/volume12/collobert11a/collobert11a.pdf
[4 ๐Ÿ“š] Phoneme Recognition using Time Delay Neural Networks: https://www.cs.toronto.edu/~fritz/absps/waibelTDNN.pdf
[5 ๐Ÿ“š] Efficient Estimation of words in vector space: arxiv.org/pdf/1301.3781.pdf
[6 ๐Ÿ“š] GloVe: https://nlp.stanford.edu/pubs/glove.pdf
[7 ๐Ÿ“š] Enriching word vectors with subword information: arxiv.org/pdf/1607.04606.pdf
[8 ๐Ÿ“š] A Convolution Neural Network for modeling sentences: arxiv.org/pdf/1404.2188.pdf
[9 ๐Ÿ“š] Learning Internal Representations by error propagation: https://apps.dtic.mil/dtic/tr/fulltext/u2/a164453.pdf
[10 ๐Ÿ“š] Sequence Modeling (from the deep learning book): deeplearningbook.org/contents/rnn.html#pf1d
[11 ๐Ÿ“š] Long Short Term Memory: http://www.bioinf.jku.at/publications/older/2604.pdf
[12 ๐Ÿ“š] Colah's blog to understanding LSTM: colah.github.io/posts/2015-08-Understanding-LSTMs
[13 ๐Ÿ“š] Training Recurrent Neural Networks (PhD Thesis): cs.utoronto.ca/~ilya/pubs/ilya_sutskever_phd_thesis.pdf
[14 ๐Ÿ“š] Deep contextualized word representations: arxiv.org/pdf/1802.05365.pdf
[15 ๐Ÿ“š] Attention is all you need: arxiv.org/pdf/1706.03762.pdf
[16๐Ÿ“š] BERT: arxiv.org/pdf/1810.04805.pdf
[17 ๐Ÿ“š] Improving language understanding by generative pretraining: s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf
[18 ๐Ÿ“š] Language Models are multi task learners:d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf
[19 ๐Ÿ“š] Language models are few shot learners: arxiv.org/pdf/2005.14165.pdf
[20 ๐Ÿ“š] Sentence BERT: arxiv.org/pdf/1908.10084.pdf
[21 ๐Ÿ“š] ChatGPT blog: openai.com/blog/chatgpt
[22 ๐Ÿ“š] Llama-2: scontent-lax3-1.xx.fbcdn.net/v/t39.2365-6/10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=cvmnxtGA6rQAX9vXmYA&_nc_ht=scontent-lax3-1.xx&oh=00_AfAfUpzEkX3nP_ptxAOoqKL2pO7s0fzAsAyYEXPFmg1fLQ&oe=64E1FF7F

MY VIDEOS ON THESE TOPICS
๐Ÿ’ป Convolution in NLP: youtube.com/watch?v=AGWieLbom_g
๐Ÿ’ป LSTM: youtube.com/watch?v=QciIcRxJvsM
๐Ÿ’ป Transformers: youtube.com/watch?v=TQQlZhbC5ps
๐Ÿ’ป Coding Transformers (playlist): youtube.com/playlist?list=PLTl9hO2Oobd97qfWC40gOSU8C0iu0m2l4
๐Ÿ’ป BERT: youtube.com/watch?v=xI0HHN5XKDo
๐Ÿ’ป GPT: youtube.com/watch?v=3IweGfgytgY
๐Ÿ’ป ChatGPT: youtube.com/watch?v=NpmnWgQgcsA
๐Ÿ’ป Llama: youtube.com/watch?v=yZ9jkgN2xHQ&t=220s

MATH COURSES (7 day free trial)
๐Ÿ“• Mathematics for Machine Learning: imp.i384100.net/MathML
๐Ÿ“• Calculus: imp.i384100.net/Calculus
๐Ÿ“• Statistics for Data Science: imp.i384100.net/AdvancedStatistics
๐Ÿ“• Bayesian Statistics: imp.i384100.net/BayesianStatistics
๐Ÿ“• Linear Algebra: imp.i384100.net/LinearAlgebra
๐Ÿ“• Probability: imp.i384100.net/Probability

OTHER RELATED COURSES (7 day free trial)
๐Ÿ“• โญ Deep Learning Specialization: imp.i384100.net/Deep-Learning
๐Ÿ“• Python for Everybody: imp.i384100.net/python
๐Ÿ“• MLOps Course: imp.i384100.net/MLOps
๐Ÿ“• Natural Language Processing (NLP): imp.i384100.net/NLP
๐Ÿ“• Machine Learning in Production: imp.i384100.net/MLProduction
๐Ÿ“• Data Science Specialization: imp.i384100.net/DataScience
๐Ÿ“• Tensorflow: imp.i384100.net/Tensorflow

#chatgpt #deeplearning #machinelearning #bert #gpt
75 years of Language ModelsLLM Self-Taught Reasoning - Explained!Multi Armed Bandits - Reinforcement Learning Explained!LSTM gates explained in 60 seconds #deeplearning #machinelearningEvolution of back propagation: Who invented it?Region Proposals - Explained!Time Series Modeling   trend and seasonality #deeplearning #machinelearningQ-learning - Explained!NLP with Neural Networks | ngram to LLMsFeed forward networks in TransformersPlaylist to Code Transformer ModelFull Timeline of Computer Vision & Neural Networks
CodeEmporium |

75 years of Language Models

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER