Uploaded February 2026 | Updated September 2026, 2 weeks ago
Arize AX Enablement session on building evaluators! In this hands-on session, we show you how to create LLM-as-a-Judge and code-based evaluators, scope them across spans, traces, and sessions, and apply filters to evaluate on exactly the data that matters. Designed for both technical and non-technical attendees - this session focuses on real examples that make evaluators approachable and actionable for everyone.
Learn how OpenTelemetry tracing works for AI apps—spans, traces, sessions, exporters, span processors, and tracer providers—and how OpenInference semantic conventions + auto-instrumentation make LLM/agent workflows observable in Arize AX. We close with advanced topics like context propagation, sampling, and OTEL collectors.
0:00 Intro
0:49 Why observability (trust + silent failures)
4:07 OpenTelemetry overview
7:17 OpenInference overview
8:21 Signals (traces, metrics, & logs)
10:15 Span structure
16:31 OTel components (resource/exporter/processor/provider)
18:17 Resource
21:48 Exporters
31:32 Span processors
35:05 Tracer provider
38:53 Register function (arize.otel)
39:24 ArizeRoutingProcessor
41:25 OpenInference Semantic Conventions
44:59 How to Instrument
45:51 Autoinstrumentation
46:58 Manual instrumentation
51:17 OpenInference Context Managers
52:15 Hybrid instrumentation
53:54 Custom Span Processors
57:26 Context propagation
1:05:36 Sampling
1:12:43 Collector
Speakers:
Sofia Jakovcevic: linkedin.com/in/sofia-jakovcevic
Lucas Moehlenbrock: linkedin.com/in/lucas-moehlenbrock-b42174130
Arize docs:
Tracing overview: arize.com/docs/ax/observe/tracing
Quickstart tracing: arize.com/docs/ax/quickstarts/quickstart-tracing
OpenTelemetry integration (arize-otel): arize.com/docs/ax/integrations/opentelemetry/opentelemetry-arize-otel
OpenInference helpers: arize.com/docs/ax/observe/tracing/configure-tracing-options/instrument-with-openinference-helpers
Advanced OTEL examples: arize.com/docs/ax/observe/tracing/configure/advanced-tracing-otel-examples
Arize community events:
arize.com/community
Arize AX Enablement session on building evaluators! In this hands-on session, we show you how to create LLM-as-a-Judge and code-based evaluators, scope them across spans, traces, and sessions, and apply filters to evaluate on exactly the data that matters. Designed for both technical and non-technical attendees - this session focuses on real examples that make evaluators approachable and actionable for everyone.
Learn how OpenTelemetry tracing works for AI apps—spans, traces, sessions, exporters, span processors, and tracer providers—and how OpenInference semantic conventions + auto-instrumentation make LLM/agent workflows observable in Arize AX. We close with advanced topics like context propagation, sampling, and OTEL collectors.
0:00 Intro
0:49 Why observability (trust + silent failures)
4:07 OpenTelemetry overview
7:17 OpenInference overview
8:21 Signals (traces, metrics, & logs)
10:15 Span structure
16:31 OTel components (resource/exporter/processor/provider)
18:17 Resource
21:48 Exporters
31:32 Span processors
35:05 Tracer provider
38:53 Register function (arize.otel)
39:24 ArizeRoutingProcessor
41:25 OpenInference Semantic Conventions
44:59 How to Instrument
45:51 Autoinstrumentation
46:58 Manual instrumentation
51:17 OpenInference Context Managers
52:15 Hybrid instrumentation
53:54 Custom Span Processors
57:26 Context propagation
1:05:36 Sampling
1:12:43 Collector
Speakers:
Sofia Jakovcevic: linkedin.com/in/sofia-jakovcevic
Lucas Moehlenbrock: linkedin.com/in/lucas-moehlenbrock-b42174130
Arize docs:
Tracing overview: arize.com/docs/ax/observe/tracing
Quickstart tracing: arize.com/docs/ax/quickstarts/quickstart-tracing
OpenTelemetry integration (arize-otel): arize.com/docs/ax/integrations/opentelemetry/opentelemetry-arize-otel
OpenInference helpers: arize.com/docs/ax/observe/tracing/configure-tracing-options/instrument-with-openinference-helpers
Advanced OTEL examples: arize.com/docs/ax/observe/tracing/configure/advanced-tracing-otel-examples
Arize community events:
arize.com/community
