Uploaded August 2025 | Updated September 2026, 2 weeks ago

---

Context engineering is going to play a big role in AI in the future 🚀

Most progress so far has come from putting knowledge into model weights through pre-training, fine-tuning, and reinforcement learning. We’ll continue to see advances there but we’re also going to see major improvements in how knowledge is fed into context and managed externally.

🎥 Robert Nishihara explains why context engineering will matter and why we’re about to see a tremendous amount of progress on this dimension.

Here are 3 benefits of storing knowledge external to the model:

1️⃣ A way to remove or fix incorrect knowledge
If everything is baked into weights, unlearning is extremely difficult. With external storage, you can simply update or delete bad information.

2️⃣ Interpretability - you can see what the model knows
External knowledge is transparent. You can directly inspect what a model believes, instead of guessing what’s hidden in billions of parameters.

3️⃣ A way to use reasoning at training time
Instead of discarding reasoning after inference, we can capture it, curate it, and feed it back into learning.

The TLDR: context engineering can help move us toward systems that can update, refine, and self-correct as they encounter new data ⚙️