Uploaded December 2025 | Updated September 2026, 18 minutes ago
Day 3/42: What is a Token?
Yesterday, we said an LLM is just a powerful autocomplete system.
Today, we define what it’s actually predicting: tokens.
A token isn’t always a word.
It can be part of a word, punctuation, or even a space.
“Running” might become: `runn` + `ing`
And every question you ask is first chopped into these pieces.
Why this matters:
tokens are the currency of LLMs. Cost, speed, context limits, and even failures all start here.
Missed Day 2? That’s where this clicks.
Tomorrow, we’ll see how tokens get meaning instead of just being numbers.
I’m Louis-François, PhD dropout, now CTO & co-founder at Towards AI. Follow me for tomorrow’s no-BS AI roundup 🚀
#LLM #Tokens #AIExplained #short
Day 3/42: What is a Token?
Yesterday, we said an LLM is just a powerful autocomplete system.
Today, we define what it’s actually predicting: tokens.
A token isn’t always a word.
It can be part of a word, punctuation, or even a space.
“Running” might become: `runn` + `ing`
And every question you ask is first chopped into these pieces.
Why this matters:
tokens are the currency of LLMs. Cost, speed, context limits, and even failures all start here.
Missed Day 2? That’s where this clicks.
Tomorrow, we’ll see how tokens get meaning instead of just being numbers.
I’m Louis-François, PhD dropout, now CTO & co-founder at Towards AI. Follow me for tomorrow’s no-BS AI roundup 🚀
#LLM #Tokens #AIExplained #short
