How to Turn Failing Traces Into a Better Agent | Ep. 10 @arizeai
How to Turn Failing Traces Into a Better Agent | Ep. 10  @arizeai
Uploaded July 2026 | Updated November 2026, 3 days ago
Learn how to turn the failures you've found into a regression dataset, then prove your fix actually works instead of just hoping it does. Finding bugs is only half the job.

A prompt tweak that "feels" better isn't proof. A controlled experiment against a saved regression set is what tells you the score change is real, not noise.

Watch this to learn:
β€’ How to save failing spans as a dataset and use it as a regression test set
β€’ How to run controlled experiments so score changes are attributable to your change, not chance
β€’ Where to intervene: data, prompt, or model, and how eval-driven development flips the usual workflow

Chapters:
00:00 Using evals to improve the agent
00:36 Save failures as a dataset, your regression set
02:39 Let Claude propose the fix, grounded in explanations
03:55 Running a controlled experiment
04:48 Comparing results example by example
06:16 How many samples do you need?
06:29 Where to intervene: data, prompt, or model
07:20 Eval-driven development
08:22 Next: production and online evals

πŸ‘‰ Sign up for free: app.arize.com/auth/login?utm_source=youtube&utm_medium=organic_social&utm_campaign=arize_ax_getting_started
πŸ”— Learn more about Arize AX: arize.com
πŸ““ Colab notebook for the series: colab.research.google.com/drive/1dViThD0kJjbqtDGE7-ciBYIRwV-HBN1T
πŸ“š Docs: docs.arize.com
πŸ”” Subscribe for weekly content on LLMs, agents, and evaluation: youtube.com/@arizeai?sub_confirmation=1

#ArizeAX #AgentEvaluation #LLMEvaluation
How to Turn Failing Traces Into a Better Agent | Ep. 10How AI Ops Teams Will Manage Fleets of Agents | ArizeArize AX: Alyx Demo 2026How a Simple To-Do List Fix Boosted Agent Completion RatesEveryone Is an AI Agent Builder (If You Let Them) | CrewAI | Arize Observe 2026OpenTelemetry GenAI Semantic Conventions: Native Support in Arize AXFind the traces that matter with Phoenix and PXIHow to Manage LLM Context Windows for AI AgentsWhy AI Agents Fail at Tasks They Already SolvedScaling Reinforcement Learning for AI Agents with Ray | Anyscale | Arize Observe 2026Humans vs. AI Agents: Who Should Handle What in Production?Prompt Learning: How We Made Claude Code 20% Better Without Changing the Model
Arize AI |

How to Turn Failing Traces Into a Better Agent | Ep. 10

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER