Uploaded October 2026 | Updated November 2026, 2 days ago
Harrison Chase, Co-Founder and CEO of LangChain, splits an agent into three parts, the model, the harness, and the context, and explains how each one can improve over time: the model's weights through supervised fine-tuning with LangSmith Fine-Tuning and its smithtune CLI, or through RL for more complex tasks; the harness through LangSmith Engine, a deep agent that runs over traces, clusters issues, and proposes fixes with a human in the loop; and the context through memory, where he points to Hex and Profound as companies he thinks do this well and to LangChain's open source OpenWiki project. He argues that traces and evals are the common foundation of all three loops, which is why LangChain built SmithDB and the new LangSmith AI Search for trace data, then covers what makes evals hard (realistic inputs, defining what good looks like, simulation environments) and how red teaming in LangSmith Engine looks for agent mistakes proactively. He closes with what he calls a North Star vision: an agent that notices its own mistakes, creates its own eval data points, and suggests its own fixes.
This session was captured at Interrupt London, the agent conference by LangChain.
Chapters:
0:00 What is an agent, and how do you improve it?
3:22 When should you fine-tune the model behind your agent?
5:52 How LangSmith Engine proposes fixes to your harness
7:13 Why context is the easiest part of an agent to improve
8:42 How OpenWiki gives coding agents a brain that updates
9:11 Why traces and evals are the foundation of improvement
10:44 Why LangChain built SmithDB and AI Search for traces
12:02 Why evals are the new PRD
12:50 Why is building evals so hard?
14:57 How to run evals without hitting production systems
16:01 How red teaming finds agent mistakes ahead of time
16:54 How do you judge correctness without a universal judge?
18:54 The flywheel that improves your agent over time
19:28 What if the agent ran its own improvement loop?
Resources:
→ LangSmith Engine: langchain.com/langsmith/engine
→ LangSmith Fine-Tuning (smithtune): docs.langchain.com/langsmith/smithtune
→ OpenWiki on GitHub: github.com/langchain-ai/openwiki
→ LangSmith: langchain.com/langsmith
→ Interrupt, the agent conference by LangChain: interrupt.langchain.com
Harrison Chase, Co-Founder and CEO of LangChain, splits an agent into three parts, the model, the harness, and the context, and explains how each one can improve over time: the model's weights through supervised fine-tuning with LangSmith Fine-Tuning and its smithtune CLI, or through RL for more complex tasks; the harness through LangSmith Engine, a deep agent that runs over traces, clusters issues, and proposes fixes with a human in the loop; and the context through memory, where he points to Hex and Profound as companies he thinks do this well and to LangChain's open source OpenWiki project. He argues that traces and evals are the common foundation of all three loops, which is why LangChain built SmithDB and the new LangSmith AI Search for trace data, then covers what makes evals hard (realistic inputs, defining what good looks like, simulation environments) and how red teaming in LangSmith Engine looks for agent mistakes proactively. He closes with what he calls a North Star vision: an agent that notices its own mistakes, creates its own eval data points, and suggests its own fixes.
This session was captured at Interrupt London, the agent conference by LangChain.
Chapters:
0:00 What is an agent, and how do you improve it?
3:22 When should you fine-tune the model behind your agent?
5:52 How LangSmith Engine proposes fixes to your harness
7:13 Why context is the easiest part of an agent to improve
8:42 How OpenWiki gives coding agents a brain that updates
9:11 Why traces and evals are the foundation of improvement
10:44 Why LangChain built SmithDB and AI Search for traces
12:02 Why evals are the new PRD
12:50 Why is building evals so hard?
14:57 How to run evals without hitting production systems
16:01 How red teaming finds agent mistakes ahead of time
16:54 How do you judge correctness without a universal judge?
18:54 The flywheel that improves your agent over time
19:28 What if the agent ran its own improvement loop?
Resources:
→ LangSmith Engine: langchain.com/langsmith/engine
→ LangSmith Fine-Tuning (smithtune): docs.langchain.com/langsmith/smithtune
→ OpenWiki on GitHub: github.com/langchain-ai/openwiki
→ LangSmith: langchain.com/langsmith
→ Interrupt, the agent conference by LangChain: interrupt.langchain.com




