Uploaded November 2026 | Updated November 2026, 4 days ago
What happens when your AI agents score above 90 on evals, but their production traces tell a different story? At Uber, teams discovered agent failures that high evaluation scores missed, including a voice agent that redirected a rider after picking up a conversation happening in the background.
Watch Aayush Agrawal, AI Product Lead at Uber, share how Uber uses Arize AX to connect evaluations with real agent traces and bring engineers, product managers, and designers into a shared workflow for improving AI quality.
Watch this video to learn:
- How to use agent traces to identify failures that high LLM evaluation scores miss
- Why AI agent evaluation and instrumentation should start during experimentation, before production
- How Uber uses Arize AX and Arize Skills to make agent evaluation accessible across engineering, product, and design
- How real-world voice agent failures inform evaluation datasets and quality criteria
Chapters:
00:00 When 90+ eval scores don't match agent traces
00:27 How AI agent observability is evolving
00:38 Why evaluation should start before production
01:20 Moving beyond evals as a launch checklist
01:44 Bringing product and operations into evaluation
02:09 Using Arize Skills from the IDE
02:25 Giving PMs and designers ownership of agent quality
02:39 Debugging real-world voice agent failures
03:09 Why teams need to continuously improve evals
03:27 How Uber works with Arize at scale
03:46 What's next for AI agent evaluation at Uber
Resources:
đź”— Arize AX: arize.com
🔬 Arize Phoenix (open source): phoenix.arize.com
đź“– AI Agent Evaluation: arize.com/ai-agents/agent-evaluation
đź“– OpenInference: github.com/Arize-ai/openinference
đź“– Phoenix documentation: docs.arize.com/phoenix
đź”” Subscribe for weekly content on LLMs, agents, and evaluation: youtube.com/@arizeai?sub_confirmation=1
#ArizeAI #LLMEvaluation #AgentObservability #Uber #AIAgents
What happens when your AI agents score above 90 on evals, but their production traces tell a different story? At Uber, teams discovered agent failures that high evaluation scores missed, including a voice agent that redirected a rider after picking up a conversation happening in the background.
Watch Aayush Agrawal, AI Product Lead at Uber, share how Uber uses Arize AX to connect evaluations with real agent traces and bring engineers, product managers, and designers into a shared workflow for improving AI quality.
Watch this video to learn:
- How to use agent traces to identify failures that high LLM evaluation scores miss
- Why AI agent evaluation and instrumentation should start during experimentation, before production
- How Uber uses Arize AX and Arize Skills to make agent evaluation accessible across engineering, product, and design
- How real-world voice agent failures inform evaluation datasets and quality criteria
Chapters:
00:00 When 90+ eval scores don't match agent traces
00:27 How AI agent observability is evolving
00:38 Why evaluation should start before production
01:20 Moving beyond evals as a launch checklist
01:44 Bringing product and operations into evaluation
02:09 Using Arize Skills from the IDE
02:25 Giving PMs and designers ownership of agent quality
02:39 Debugging real-world voice agent failures
03:09 Why teams need to continuously improve evals
03:27 How Uber works with Arize at scale
03:46 What's next for AI agent evaluation at Uber
Resources:
đź”— Arize AX: arize.com
🔬 Arize Phoenix (open source): phoenix.arize.com
đź“– AI Agent Evaluation: arize.com/ai-agents/agent-evaluation
đź“– OpenInference: github.com/Arize-ai/openinference
đź“– Phoenix documentation: docs.arize.com/phoenix
đź”” Subscribe for weekly content on LLMs, agents, and evaluation: youtube.com/@arizeai?sub_confirmation=1
#ArizeAI #LLMEvaluation #AgentObservability #Uber #AIAgents










