How to Evaluate Images with LLM-as-a-Judge | Multimodal Evals in Arize AX @arizeai
How to Evaluate Images with LLM-as-a-Judge | Multimodal Evals in Arize AX  @arizeai
Uploaded September 2026 | Updated November 2026, 2 days ago
This video walks you through the guide at arize.com/docs/ax/cookbooks/evaluate/evaluate-receipt-agents-with-image-judge.

LLM-as-a-judge evaluation is not limited to text. You can also evaluate multimodal AI applications by using images as the source of truth.

In this Arize AX walkthrough, see how to evaluate a receipt extraction agent with a multimodal LLM-as-a-judge evaluator.

The example agent takes a receipt image and extracts structured JSON, including the merchant, line items, amounts, currency, subtotal, and total. The challenge is that checking the JSON format alone does not tell you whether the extracted values actually match the image.

To solve that, we build a visual groundedness evaluator that takes the original receipt image, independently interprets its contents, and compares them against the agent’s extracted output.

In this tutorial, you’ll learn how to:

• Trace a multimodal AI application in Arize AX
• Evaluate image inputs with LLM-as-a-judge
• Use the original image as ground truth for an evaluation
• Compare structured JSON extraction against receipt contents
• Build a visual groundedness evaluator
• Filter evaluations to specific LLM spans
• Map image and output fields into an evaluator prompt
• Define grounded, not grounded, and needs review outcomes
• Run evaluations continuously on production traces
• Sample production traffic to control evaluation cost
• Use a stronger judge model to validate a cheaper extraction model
• Surface cases that should be escalated to human review

The demo also shows why multimodal judges can catch more than simple extraction mismatches. In one example, the judge notices that the individual receipt items do not add up to the reported subtotal, flagging a consistency issue even though the extracted values themselves appear plausible.

This creates a practical evaluation pattern for document extraction, receipts, invoices, forms, screenshots, and other multimodal AI workflows.

Chapters:
00:00 LLM-as-a-judge for multimodal content
00:32 Evaluating a receipt extraction agent
01:22 Why text-only evals are not enough
02:03 Building a visual groundedness evaluator
03:06 Set up the example application
04:06 Trace receipt images in Arize AX
04:56 Create the image-based LLM judge
06:10 Configure labels and span filters
07:04 Map the image and extraction fields
07:45 Run the evaluator continuously
08:10 Review grounded and failed examples
08:48 Catching inconsistencies the agent missed

🔗 Explore Arize AX: arize.com/ax
🔗 Read the Arize docs: arize.com/docs/ax
🔔 Subscribe for more videos on LLM evals, multimodal AI, agent observability, and AI engineering:
youtube.com/@arizeai?sub_confirmation=1

#LLMEvals #MultimodalAI #LLMasAJudge
How to Evaluate Images with LLM-as-a-Judge | Multimodal Evals in Arize AXWhat Happens When AI Agents Review Each Other’s Code?5 LLM and Agent Eval Mistakes That Turn Metrics Into Noise | Ep. 8Kubernetes Is Not Your Sandbox: Building Infrastructure for AI Agents | Daytona | Arize Observe 2026AI Agent Mastery Certification Course: Module 5 – RAG & Agentic RAGAI’s Next Wave: What VCs Are Betting On in 2026 | Jaya Gupta | Arize Observe 2026Stop YOLO-ing AI Observability: Building Production-Ready Agents | Arize x AWSServiceNow’s AgentArch: Benchmarking AI Agents for Enterprise WorkflowsHow to Evaluate Tool-Calling AgentsIs Your LLM Judge Right? Calibrate with Meta-Evaluation | Ep. 9Meet PXI: the AI engineering agent inside PhoenixHow to Build a Real AI Agent (Financial Analyst) with the Claude Agent SDK | Ep. 4
Arize AI |

How to Evaluate Images with LLM-as-a-Judge | Multimodal Evals in Arize AX

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER