Agent and LLM Evaluation for AI Product Managers @arizeai
Agent and LLM Evaluation for AI Product Managers  @arizeai
Uploaded November 2025 | Updated November 2026, 1 hour ago
​Great AI products don't just happen, they are the result of strong collaboration between Product Managers and Engineers who share ownership of evaluation. In this webinar, we’ll show you how PMs and engineers can build effective evals together, from early “vibe checks” to production monitoring.

​Aman Khan, Head of Product at Arize AI, will share a concrete playbook for creating alignment between product and engineering - so your next AI product launch isn’t just good, it’s trusted.

​You’ll Learn How To:
​Establish Clear Roles: Define PM and Engineer ownership from prototype to production.
Collaborate in Practice: Co-build eval prompts and run joint labeling sessions.
Future-proof launch: Create a framework for reconciling strong eval metrics with actual business outcomes.

​Topics We’ll Cover:

​The Three AI PM Archetypes
Live Co-Authoring of Evaluation Prompts
Joint Labeling & Edge Case Exploration
Debugging and Post-Launch Analysis
Connecting Evaluation Metrics to KPIs
Agent and LLM Evaluation for AI Product ManagersFixing an Agents SQL Tool Calls with Phoenix and PXILLM-as-a-Judge Evaluation: When To Use Role of COT, Reasoning & ExplanationsMCP vs. the CLI: a head-to-head evaluation of agent tool integration patternsHow AI Agents Will Debug Their Own Production CodeThe Model Knew What to Say. The Harness Failed. | Rise of the AI Engineer | Laurie VossAI Agents Need Specs, Not Prompts | George Zhang, OpenClaw | Arize Observe 2026AI Agents Certification Course: Lab 1 and 2 – Base Agent Setup & First Run + ObservabilityHow Startups Slash LLM Token Bills Without Losing QualityThe Return of the Data Scientist in AI Engineering | Hamel Husain | Arize Observe 2026Boost Claude Code performance with prompt learning - optimize your prompts automatically with evalsHow Coinbase Rebuilt Product Development Around AI Agents | Arize Observe 2026
Arize AI |

Agent and LLM Evaluation for AI Product Managers

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER