Uploaded November 2026 | Updated November 2026, 3 hours ago
What does it cost to run an LLM judge on every span?
Decision models are small, fast models built to pick from a set of choices. You pass in the input and the choices. They return each choice with a probability that it is the right one. They do not write text. They run faster than an LLM, and they are cheap enough to run on every span.
In this session we will cover:
- What decision models are, and how the input and output work
- A short demo of a decision model in action
- Using them as eval judges
- How they compare with an LLM on cost and speed
- Using them as real-time guardrails
- Where they fit, and where an LLM is still the right call
Who this is for:
AI engineers building and evaluating AI agents.
What you'll leave with:
How to read a decision model's probabilities, and how to use one as an eval judge and a real-time guardrail.
What does it cost to run an LLM judge on every span?
Decision models are small, fast models built to pick from a set of choices. You pass in the input and the choices. They return each choice with a probability that it is the right one. They do not write text. They run faster than an LLM, and they are cheap enough to run on every span.
In this session we will cover:
- What decision models are, and how the input and output work
- A short demo of a decision model in action
- Using them as eval judges
- How they compare with an LLM on cost and speed
- Using them as real-time guardrails
- Where they fit, and where an LLM is still the right call
Who this is for:
AI engineers building and evaluating AI agents.
What you'll leave with:
How to read a decision model's probabilities, and how to use one as an eval judge and a real-time guardrail.










