Uploaded December 2025 | Updated September 2026, 1 hour ago

---

Day 7/42: What Is Pre-Training?

Yesterday, we met parameters.

Today, we explain how they stop being random.

Pre-training is the long study phase of an LLM.

The model reads massive amounts of text and plays one game only:

“Predict the next token.”

Every mistake slightly adjusts the parameters.

Trillions of tiny corrections later, patterns stick.

Grammar emerges.

Facts appear.

Reasoning *seems* to happen.

Missed Day 6? Start there.

Tomorrow, we’ll see why a trained model still isn’t very helpful: base vs instruct models.

I’m Louis-François, PhD dropout, now CTO & co-founder at Towards AI. Follow me for tomorrow’s no-BS AI roundup 🚀

#PreTraining #LLM #AIExplained #short