How Uber is making evals the operating system for production AI agents with Arize @arizeai
How Uber is making evals the operating system for production AI agents with Arize  @arizeai
Uploaded November 2026 | Updated November 2026, 4 days ago
What happens when your AI agents score above 90 on evals, but their production traces tell a different story? At Uber, teams discovered agent failures that high evaluation scores missed, including a voice agent that redirected a rider after picking up a conversation happening in the background.

Watch Aayush Agrawal, AI Product Lead at Uber, share how Uber uses Arize AX to connect evaluations with real agent traces and bring engineers, product managers, and designers into a shared workflow for improving AI quality.

Watch this video to learn:
- How to use agent traces to identify failures that high LLM evaluation scores miss
- Why AI agent evaluation and instrumentation should start during experimentation, before production
- How Uber uses Arize AX and Arize Skills to make agent evaluation accessible across engineering, product, and design
- How real-world voice agent failures inform evaluation datasets and quality criteria

Chapters:
00:00 When 90+ eval scores don't match agent traces
00:27 How AI agent observability is evolving
00:38 Why evaluation should start before production
01:20 Moving beyond evals as a launch checklist
01:44 Bringing product and operations into evaluation
02:09 Using Arize Skills from the IDE
02:25 Giving PMs and designers ownership of agent quality
02:39 Debugging real-world voice agent failures
03:09 Why teams need to continuously improve evals
03:27 How Uber works with Arize at scale
03:46 What's next for AI agent evaluation at Uber

Resources:
đź”— Arize AX: arize.com
🔬 Arize Phoenix (open source): phoenix.arize.com
đź“– AI Agent Evaluation: arize.com/ai-agents/agent-evaluation
đź“– OpenInference: github.com/Arize-ai/openinference
đź“– Phoenix documentation: docs.arize.com/phoenix

đź”” Subscribe for weekly content on LLMs, agents, and evaluation: youtube.com/@arizeai?sub_confirmation=1

#ArizeAI #LLMEvaluation #AgentObservability #Uber #AIAgents
How Uber is making evals the operating system for production AI agents with ArizeMulti-Agent Observability: Debugging Agent-to-Agent Communication | Band | Arize Observe 2026AI Agents in Production: From Demos to Durable ROI | CVS Health | Arize Observe 2026AI Agent Mastery Certification Course: Module 3 – Agent Architectures & FrameworksRules Belong in Code, Not Prompts: How to Build Production AI AgentsEU AI Act: How To Create a Dashboard To Monitor ComplianceStop Paying More for Smarter LLMsAligning LLM Evaluators with Human Annotations (using Mastra agents)LLM as a Judge 102: Meta EvaluationFrom AI Coding Agents to the Software Factory | Factory AI | Arize Observe 2026One AI Question - why do you use an LLM to evaluate another LLM with Ankur DuggalHow to Build Reliable Long-Running AI Agents | Arize x Temporal
Arize AI |

How Uber is making evals the operating system for production AI agents with Arize

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER