LLMs Explained: Tokens, Embeddings, Transformers and More @syntaxfm
LLMs Explained: Tokens, Embeddings, Transformers and More  @syntaxfm
Uploaded July 2026 | Updated November 2026, 3 days ago
In this video CJ explains chat bots, neural networks, tokenization, embeddings, transformers and more. He shows real working code for everything and gives historical context along the way. View the code here: github.com/w3cj/how-llms-work

00:00 - intro
01:54 - history of chatbots
04:21 - chat bot code
06:49 - black box thinking
07:41 - what are neural networks?
08:33 - history of neural networks
10:27 - XOR neural network code
15:35 - what is tokenization?
16:32 - history of tokenization
17:55 - tokenizer code
21:37 - what are embeddings?
22:04 - history of embeddings
23:33 - what is a vector?
24:39 - embedding code
30:48 - history of transformers
32:33 - what is a transformer?
33:11 - what is self attention?
34:25 - what is a feed forward network?
34:37 - what are stacked transformer blocks?
41:37 - what are softmax, temperature and top-p?
42:54 - the autoregressive loop
43:28 - the context window
44:45 - the full picture
45:26 - what is fine-tuning
46:39 - RLHF
47:24 - tool calling
48:36 - the AI summit
49:51 - final thoughts

Claude Shannon | en.wikipedia.org/wiki/Claude_Shannon
Betty Shannon | en.wikipedia.org/wiki/Betty_Shannon
Prediction and Entropy of Printed English (1951) | https://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf

Alan Turing | en.wikipedia.org/wiki/Alan_Turing
Turing Test | en.wikipedia.org/wiki/Computing_Machinery_and_Intelligence
Joseph Weizenbaum | en.wikipedia.org/wiki/Joseph_Weizenbaum
ELIZA | en.wikipedia.org/wiki/ELIZA
ELIZA - Weizenbaum (1966) | dl.acm.org/doi/10.1145/365153.365168
ELIZA Archaeology Project | sites.google.com/view/elizaarchaeology/blog/3-weizenbaums-secretary
Computer Power and Human Reason | archive.org/details/computerpowerhum0000weiz_v0i3
Kenneth Colby | en.wikipedia.org/wiki/Kenneth_Colby
PARRY | en.wikipedia.org/wiki/PARRY
Artificial Paranoia (1971) (PDF) | https://courses.cs.umbc.edu/graduate/671/fall20/resources/colby_71.pdf
ELIZA connected to PARRY - RFC 439 | datatracker.ietf.org/doc/html/rfc439
ALICE | en.wikipedia.org/wiki/Artificial_Linguistic_Internet_Computer_Entity
SmarterChild | en.wikipedia.org/wiki/SmarterChild

McCulloch & Pitts (1943) | en.wikipedia.org/wiki/A_Logical_Calculus_of_the_Ideas_Immanent_in_Nervous_Activity
Frank Rosenblatt | en.wikipedia.org/wiki/Frank_Rosenblatt
Perceptron | en.wikipedia.org/wiki/Perceptron
Rosenblatt's Perceptron - Cornell | https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon
New Navy Device Learns by Doing - NYT (1958) | timesmachine.nytimes.com/timesmachine/1958/07/08/issue.html
Perceptrons | en.wikipedia.org/wiki/Perceptrons_(book)

Backpropagation - Rumelhart, Hinton & Williams (1986) | nature.com/articles/323533a0

BPE - Philip Gage (1994) (PDF) | derczynski.com/papers/archive/BPE_Gage.pdf
Byte Pair Encoding | en.wikipedia.org/wiki/Byte-pair_encoding
BPE for NMT - Sennrich, Haddow & Birch (2015) | arxiv.org/abs/1508.07909

Gottlob Frege | en.wikipedia.org/wiki/Gottlob_Frege
Foundations of Arithmetic - Frege (1884) | archive.org/details/foundationsofari00fregrich
Context Principle | en.wikipedia.org/wiki/Context_principle
J.R. Firth | en.wikipedia.org/wiki/John_Rupert_Firth
Studies in Linguistic Analysis - Firth (1957) | archive.org/details/studiesinlinguis0000vari
Word2Vec - Mikolov et al. (2013) | arxiv.org/abs/1301.3781
Word2Vec Negative Sampling - Mikolov et al. (2013) | arxiv.org/abs/1310.4546

Attention Mechanism - Bahdanau, Cho & Bengio (2014) | arxiv.org/abs/1409.0473
GNMT - Wu et al. (2016) | arxiv.org/abs/1609.08144
Xavier Initialization - Glorot & Bengio (2010) | https://proceedings.mlr.press/v9/glorot10a.html
Attention Is All You Need - Vaswani et al. (2017) | arxiv.org/abs/1706.03762
Transformer | en.wikipedia.org/wiki/Transformer_(deep_learning)

Common Crawl | en.wikipedia.org/wiki/Common_Crawl
RLHF | en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback

Dartmouth Workshop (1956) | en.wikipedia.org/wiki/Dartmouth_workshop
Dartmouth AI Proposal (1955) | https://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html

Mamba - Gu & Dao (2023) | arxiv.org/abs/2312.00752
xLSTM - Beck et al. (2024) | arxiv.org/abs/2405.04517
Jamba - Lieber et al. (2024) | arxiv.org/abs/2403.19887
JEPA - Yann LeCun (2022) | openreview.net/pdf?id=BZ5a1r-kVsf

----------------------------------------------------------------------------

http://www.syntax.fm

Brought to you by sentry.io/syntax

#LLM #machinelearning #programming #explained #javascript #typescript #syntax #syntaxfm
LLMs Explained: Tokens, Embeddings, Transformers and MoreRuby on Rails is Dead ⟡ Zuck cooked with Muse ⟡ upm is a tiny npm replacement ⌁ Syntax Weekly ⌁Jailbreaking a $50 photo frameDoes Code Quality Matter Anymore?Token embeddings explained. #llm #explainer #programing #mlOpen Weight AI Models Explained for EveryoneWTF is a Software Factory?SpaceTime DB 2.0 is 1000x Faster... (so they say) - and more! | 3 Devs - 3 ThingsWhat’s Missing From the Web Platform?Cloudflare Acquires VoidZeroTHIS is how to expose your apps to AI: WebMCPShai Hulud is back.
Syntax |

LLMs Explained: Tokens, Embeddings, Transformers and More

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER