Using Annotations to Build an Eval-Driven LLM Development Pipeline @arizeai
Using Annotations to Build an Eval-Driven LLM Development Pipeline  @arizeai
Uploaded July 2025 | Updated September 2026, 3 weeks ago
In this tutorial, you’ll learn how to build a custom human annotation interface for Phoenix using Lovable and use those annotations to run experiments and evaluate your application.

A custom annotation UI makes it easy to collect structured human feedback on traces directly in Phoenix, enabling faster iteration and improvement of your LLM systems. By establishing this feedback loop, you can effectively monitor and enhance your application’s performance.

Find the notebook here: arize.com/docs/phoenix/cookbook/tracing-and-annotations/using-human-annotations-for-eval-driven-development
Join our community slack: arize.com/community
Get started with Phoenix for free: app.arize.com/auth/phoenix/signup
More on Phoenix annotations: arize.com/docs/phoenix/tracing/features-tracing/how-to-annotate-traces
Using Annotations to Build an Eval-Driven LLM Development PipelineServiceNow’s AgentArch: Benchmarking AI Agents for Enterprise WorkflowsHow to Evaluate Tool-Calling AgentsIs Your LLM Judge Right? Calibrate with Meta-Evaluation | Ep. 9Building and Scaling ProductsElastic AI - Walking Your Way to PhoenixMeet PXI: the AI engineering agent inside PhoenixHow to Build a Real AI Agent (Financial Analyst) with the Claude Agent SDK | Ep. 4Stop Vibe-Testing Your AI Agents: How to Actually Run Evals (in 25 Minutes)Stop Blaming the Model: Fixing the AI Product Bottleneck | Rise of the AI Engineer | Hamel HusainFrom Build to Production: Engineering Reliable AI Agents with Google and ArizeHow Cursor Uses AI Agents to Build Cursor | Arize Observe 2026
Arize AI |

Using Annotations to Build an Eval-Driven LLM Development Pipeline

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER