Uploaded August 2025 | Updated September 2026, 2 weeks ago
This tutorial shows you how to run session-level evaluations on conversations with an AI tutor using Arize. Session-level evaluations provide a holistic view of entire interactions, enabling you to assess broader patterns and answer high-level questions about user experience and system performance.
Resources:
Notebook: arize.com/docs/ax/cookbooks/evaluation/session-level-evaluations-for-an-ai-tutor
Make a free Arize account: arize.com/sign-up
Arize Community Slack: arize.com/community
Arize Docs: arize.com/docs/ax
See our explainer on sessions, traces, spans in LLM observability: arize.com/blog/llm-observability-for-ai-agents-and-applications
This tutorial covers how to:
Agent tracing for multi-turn AI tutor conversations
Aggregate spans into structured sessions with truncation support
Evaluate sessions across multiple dimensions (Correctness, Goal Completion, Frustration)
Format evaluation outputs to match Arize's schema
Log results back to Arize for monitoring and analysis
Background:
A session captures the entire dialogue between a user and your app—the full journey, not just isolated spans. It reflects how people actually use your product.
Evaluating at the session level unlocks a broader view. You can measure things like frustration, context retention, and whether the user’s goal was achieved—insights you simply can’t get from span-level evaluations alone.
We built an AI tutor and turned its traces into complete sessions. Then, using Arize, we ran session-level evaluations that assess the whole conversation, not just snippets.
This tutorial shows you how to run session-level evaluations on conversations with an AI tutor using Arize. Session-level evaluations provide a holistic view of entire interactions, enabling you to assess broader patterns and answer high-level questions about user experience and system performance.
Resources:
Notebook: arize.com/docs/ax/cookbooks/evaluation/session-level-evaluations-for-an-ai-tutor
Make a free Arize account: arize.com/sign-up
Arize Community Slack: arize.com/community
Arize Docs: arize.com/docs/ax
See our explainer on sessions, traces, spans in LLM observability: arize.com/blog/llm-observability-for-ai-agents-and-applications
This tutorial covers how to:
Agent tracing for multi-turn AI tutor conversations
Aggregate spans into structured sessions with truncation support
Evaluate sessions across multiple dimensions (Correctness, Goal Completion, Frustration)
Format evaluation outputs to match Arize's schema
Log results back to Arize for monitoring and analysis
Background:
A session captures the entire dialogue between a user and your app—the full journey, not just isolated spans. It reflects how people actually use your product.
Evaluating at the session level unlocks a broader view. You can measure things like frustration, context retention, and whether the user’s goal was achieved—insights you simply can’t get from span-level evaluations alone.
We built an AI tutor and turned its traces into complete sessions. Then, using Arize, we ran session-level evaluations that assess the whole conversation, not just snippets.
