Uploaded September 2026 | Updated November 2026, 21 hours ago
Learn more about AI Observability here → https://ibm.biz/~1VvjeyWZi
AI agents fail in ways traditional monitoring can't see. Legare Kerrison explains how MLflow helps trace, evaluate, and monitor multi-agent systems using observability and OpenTelemetry. Learn how tracing, LLM judges, and evaluation workflows improve AI reliability.
00:00 - Why Traditional Monitoring Fails for Multi-Agent AI Systems
00:54 - Common AI Agent Failures You Can't See in Your Dashboard
01:14 - Introducing LLM Observability with MLflow Tracing
02:03 - Mortgage AI Agent Example: Understanding Multi-Agent Workflows
03:02 - Silent Failures, Latency Bottlenecks & Context Overflow Explained
03:56 - How MLflow Traces Every Agent, Tool Call, and LLM Interaction
04:56 - Using LLM-as-a-Judge to Evaluate AI Agent Quality
06:03 - Prompt Registry, Version Control & AI Governance Best Practices
06:23 - 4 Production-Ready MLflow Tips for Enterprise Deployments
08:18 - OpenTelemetry, LangChain Autologging & End-to-End Trace Visibility
AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~jGvnp6nxg
AI was used in the creation of the transcript and metadata for this video.
#mlflow #aiagents #llm #aimonitoring
---------------------------------------------------------------------------------------------------------
Find us on YouTube:
🔵 IBM Technology: youtube.com/@IBMTechnology
🔵 IBM: youtube.com/@IBM
🔵 IBM Developer: youtube.com/@IBMDeveloperAdvocates
Learn more about AI Observability here → https://ibm.biz/~1VvjeyWZi
AI agents fail in ways traditional monitoring can't see. Legare Kerrison explains how MLflow helps trace, evaluate, and monitor multi-agent systems using observability and OpenTelemetry. Learn how tracing, LLM judges, and evaluation workflows improve AI reliability.
00:00 - Why Traditional Monitoring Fails for Multi-Agent AI Systems
00:54 - Common AI Agent Failures You Can't See in Your Dashboard
01:14 - Introducing LLM Observability with MLflow Tracing
02:03 - Mortgage AI Agent Example: Understanding Multi-Agent Workflows
03:02 - Silent Failures, Latency Bottlenecks & Context Overflow Explained
03:56 - How MLflow Traces Every Agent, Tool Call, and LLM Interaction
04:56 - Using LLM-as-a-Judge to Evaluate AI Agent Quality
06:03 - Prompt Registry, Version Control & AI Governance Best Practices
06:23 - 4 Production-Ready MLflow Tips for Enterprise Deployments
08:18 - OpenTelemetry, LangChain Autologging & End-to-End Trace Visibility
AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~jGvnp6nxg
AI was used in the creation of the transcript and metadata for this video.
#mlflow #aiagents #llm #aimonitoring
---------------------------------------------------------------------------------------------------------
Find us on YouTube:
🔵 IBM Technology: youtube.com/@IBMTechnology
🔵 IBM: youtube.com/@IBM
🔵 IBM Developer: youtube.com/@IBMDeveloperAdvocates










