Inside Clays Eval Stack: 300M Agent Runs, One LangSmith Pipeline @LangChain
Inside Clays Eval Stack: 300M Agent Runs, One LangSmith Pipeline  @LangChain
Uploaded August 2026 | Updated November 2026, 1 day ago
Jeff Barg (Head of AI), Vyshu Khota (ML Engineer), and Soroush Khadem (Software Engineer) at Clay break down how they built and scaled Claygent and Sculptor, Clay's go to market research and engineering agents. They cover the eval philosophy that keeps agentic changes safe to ship, why they moved to a data lake architecture to give agents first class access to Clay's data, and how a large context breakthrough from Fable changed what their agents can do with production data.

Chapters:
0:00 Clay's agent stack: Claygent and Sculptor
0:13 Meet Sculptor, Clay's go to market engineering agent
0:45 Inside Claygent: web research, first party data, and scale
1:25 Sculptor today: a hundred thousand messages a week
1:52 Sculptor for Search and the road to production quality
2:03 Why evals became non negotiable
2:47 Clay's eval priorities, from local dev to CI
4:19 The four quadrants of Clay's eval philosophy
6:00 A/B metrics and where LangSmith fits in
6:58 Closing the loop between production and offline evals
8:27 What's next: Clay as an agent interface
9:00 Why external agents create an internal flywheel
10:15 Scaling the data foundation with a data lake
11:33 Why now: Fable's large context breakthrough
12:36 The self-iterating loop Clay is building toward

Resources:
→ LangSmith: langchain.com/langsmith
→ LangChain: langchain.com
→ LangChain Academy: academy.langchain.com
Inside Clays Eval Stack: 300M Agent Runs, One LangSmith PipelineNot Every Bug Engine Finds Is Worth FixingKeep model costs in check with LangSmith LLM GatewayBuilding a Harness with JevThe Agent for Your Agent.How Benchlings Team Actually Looks at AI Traces | Max AgencyBuilding Jarvis: JPMC’s Agent Building Assembly LineInside Managed Deep Agents: mda CLI, Context Hub, and Harbor Evals | Interrupt LondonWhat are Deep Agents? Explained fastBuild an Online LLM-as-a-Judge with the LangSmith CLI and a Coding AgentBuilding Docs for Agents, Not Humans: Inside OpenWikiWhy Own Your Intelligence? Harrison Chase Keynote | Interrupt London
LangChain |

Inside Clay's Eval Stack: 300M Agent Runs, One LangSmith Pipeline

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER