Building Trustworthy, High-Quality AI Agents with MLflow @Databricks
Building Trustworthy, High-Quality AI Agents with MLflow  @Databricks
Uploaded May 2026 | Updated November 2026, 3 days ago
Building AI agents presents unique challenges, as outputs can be free-form and unpredictable, often requiring specialized domain expertise to evaluate quality. This session explores how MLflow provides a unified platform to manage the full agent development life cycle. Key topics include using MLflow tracing for end-to-end observability and debugging, leveraging automated LLM judges to scale expert feedback, and employing the prompt registry for versioning and optimization. The talk also highlights the role of an AI gateway in providing essential governance through permissions, rate limits, and input guards to manage costs and data privacy.

Key Takeaways:
- Implementing end-to-end observability with MLflow tracing for step-by-step execution analysis.
- Scaling quality assessments through automated LLM-as-a-Judge evaluations and human expert alignment.
- Iteratively improving agent performance using evaluation datasets and automated prompt optimization.
- Ensuring production-grade governance and cost control with a centralized AI gateway.
Building Trustworthy, High-Quality AI Agents with MLflowDatabricks + Microsoft: Live Data, Unified Governance, and the Future of AI AgentsInside Lakebase: fully-managed serverless Postgres – Nikita Shamgunov, VP, Engineering, DatabricksWhy Halliburtons SVP of Technology Says Dont Start With AI | Sriram Srinivasan, HalliburtonTrain and fine-tune models faster with AI RuntimeHow to Build Production-Ready Apps in Replit with Lakebase Postgres | Databricks + Replit DemoDQX Studio Demo: No-Code Data Quality on DatabricksHow to Use Genie in Google Sheets: Import & Query DataHow to Govern Your AI Estate in One PlaceRecap of product announcements from Data + AI Summit 2026 | Day 1Agent Bricks: The Agent Developer Platform | OverviewHelix Accelerates Genomic Research with the Databricks Data + AI Platform
Databricks |

Building Trustworthy, High-Quality AI Agents with MLflow

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER