Uploaded November 2026 | Updated November 2026, 3 days ago
Pedro Rodriguez, Director of Engineering, ML at Airbnb, shares what his team learned building AI agents for trust, where a wrong decision affects a real guest or host and someone may later ask why it was made. He walks through the patterns they use: structured outputs with an abstain option that sends uncertain cases to a human, small subagents behind a plain rules layer, LangGraph interrupts for cases that wait days while case data lives in a separate database, MCP as the only path from agents to data, and verification of live runs with LLM judges, code checks, and a checker that runs on a different model family. He closes with why the team chose LangGraph and his main lesson: building the agents was the easy part, and knowing whether you can trust them takes the most work.
This session was captured at Interrupt New York City, the agent conference by LangChain.
Speakers:
Pedro Rodriguez, Director of Engineering, ML, Airbnb
Chapters:
0:00 What happens when one record gets two different answers?
2:03 How human agents help train Airbnb's support assistant
3:39 What does high stakes mean for trust agents?
4:35 Structured outputs and an abstain path to human review
6:58 Splitting work into subagents behind a plain rules layer
8:55 Building agents that wait days with LangGraph interrupts
10:41 Making MCP the only path from agents to data
11:46 Training the support assistant on what humans actually sent
13:05 Why evals aren't enough when labels take months
14:59 Catching inconsistent answers with a separate checker
16:24 Why Pedro's team picked LangGraph after trying others
17:25 Five takeaways, and why trust takes the most work
Resources:
→ LangGraph: langchain.com/langgraph
→ LangGraph interrupts docs: docs.langchain.com/oss/python/langgraph/interrupts
→ Introduction to LangGraph on LangChain Academy: academy.langchain.com/courses/intro-to-langgraph
→ Interrupt, the agent conference by LangChain: interrupt.langchain.com
Pedro Rodriguez, Director of Engineering, ML at Airbnb, shares what his team learned building AI agents for trust, where a wrong decision affects a real guest or host and someone may later ask why it was made. He walks through the patterns they use: structured outputs with an abstain option that sends uncertain cases to a human, small subagents behind a plain rules layer, LangGraph interrupts for cases that wait days while case data lives in a separate database, MCP as the only path from agents to data, and verification of live runs with LLM judges, code checks, and a checker that runs on a different model family. He closes with why the team chose LangGraph and his main lesson: building the agents was the easy part, and knowing whether you can trust them takes the most work.
This session was captured at Interrupt New York City, the agent conference by LangChain.
Speakers:
Pedro Rodriguez, Director of Engineering, ML, Airbnb
Chapters:
0:00 What happens when one record gets two different answers?
2:03 How human agents help train Airbnb's support assistant
3:39 What does high stakes mean for trust agents?
4:35 Structured outputs and an abstain path to human review
6:58 Splitting work into subagents behind a plain rules layer
8:55 Building agents that wait days with LangGraph interrupts
10:41 Making MCP the only path from agents to data
11:46 Training the support assistant on what humans actually sent
13:05 Why evals aren't enough when labels take months
14:59 Catching inconsistent answers with a separate checker
16:24 Why Pedro's team picked LangGraph after trying others
17:25 Five takeaways, and why trust takes the most work
Resources:
→ LangGraph: langchain.com/langgraph
→ LangGraph interrupts docs: docs.langchain.com/oss/python/langgraph/interrupts
→ Introduction to LangGraph on LangChain Academy: academy.langchain.com/courses/intro-to-langgraph
→ Interrupt, the agent conference by LangChain: interrupt.langchain.com










