LLM as a Judge: Score AI Agent Outputs with Claude (2026)

Quick answer (2026): LLM as a judge means using one language model to grade the outputs of another against a written rubric, instead of relying only on string matching or human review. It is the practical way to evaluate open-ended agent responses (summaries, tool plans, chat turns) at scale. This tutorial ships a minimal, framework-free judge harness in Python on top of the Anthropic SDK: a scored rubric, pointwise and pairwise grading, and explicit fixes for the three biases that quietly wreck LLM-judge scores (position, verbosity, and self-preference). Everything here is copy-paste runnable.

This is the subjective half of agent evaluation. For the deterministic half (asserting tool-call control flow and JSON schemas with pytest), see our companion deterministic pytest eval harness. Use both: pytest for what has a right answer, a judge for what does not.

What is LLM as a judge, and when should you use it?

A judge is a model call whose job is to read a candidate answer plus a rubric and return a structured verdict: a numeric score, a rationale, and often a pass/fail. You reach for it when the thing you want to measure has no single correct string.

Concretely, use a judge when you are grading:

• Free-text quality (helpfulness, correctness, tone, faithfulness to a source).
• Agent trajectories where many tool orderings are valid.
• Answers that must follow instructions you cannot regex for ("no medical advice", "cite the passage").

Do not use a judge when a cheaper check works. If the output is JSON, validate the schema. If there is an exact expected value, assert equality. A judge costs a model call per item and carries its own error bars, so spend it only where deterministic checks cannot reach. That triage, deterministic first, judge second, is the single most important habit in agent evals.

The three grading modes

There are three judge designs, and picking the right one is most of the battle.

Pointwise scores are the easiest to read but the noisiest: a lone model has no anchor for what a "4" means, so scores drift run to run. Pairwise is far more reliable because relative judgments ("A is better than B") are easier for a model than absolute ones, which is also why human preference datasets are collected pairwise. Reference-based sits in between and is the right call for retrieval and summarization, where you can supply the source text.

A minimal pointwise judge with Claude

You need the Anthropic Python SDK and an API key.

Install and set your key:

pip install anthropic
export ANTHROPIC_API_KEY=sk-ant-...

The design that keeps judges stable: a narrow rubric with explicit anchors for each score, and a forced JSON reply so parsing never guesses. We use claude-haiku-4-5 as the judge because it is the cheapest tier and judging is a read-then-classify task, not a generation task. Swap in claude-sonnet-5 when the rubric needs deeper reasoning.

import json
from anthropic import Anthropic

client = Anthropic()
JUDGE_MODEL = "claude-haiku-4-5"

RUBRIC = """You are a strict grader. Score the ANSWER to the QUESTION from 1 to 5.
5 = fully correct, directly answers, no unsupported claims.
4 = correct with a minor omission.
3 = partially correct or partially relevant.
2 = mostly wrong or off-topic but touches the subject.
1 = wrong, empty, or refuses without cause.
Judge only correctness and relevance. Do NOT reward length, confidence, or style.
Return ONLY JSON: {"score": , "reason": ""}."""

def judge_pointwise(question: str, answer: str) -> dict:
    msg = client.messages.create(
        model=JUDGE_MODEL,
        max_tokens=200,
        temperature=0,
        system=RUBRIC,
        messages=[{
            "role": "user",
            "content": f"QUESTION:\n{question}\n\nANSWER:\n{answer}",
        }],
    )
    text = msg.content[0].text.strip()
    return json.loads(text)

if __name__ == "__main__":
    q = "What year did the Apollo 11 crew land on the Moon?"
    print(judge_pointwise(q, "They landed in 1969."))        # -> score 5
    print(judge_pointwise(q, "Sometime in the 20th century."))  # -> score 2-3
    print(judge_pointwise(q, "I cannot help with that."))     # -> score 1

Three rubric choices are doing the real work here. temperature=0 makes the verdict as reproducible as the API allows. The explicit "do NOT reward length" line pre-empts verbosity bias (more below). And forcing a tiny JSON object means a downstream loop can grade a whole dataset without brittle text parsing. If you want a hard guarantee on the JSON shape rather than a parse-and-pray, define the reply as a tool and set tool_choice to force it, the same forcing pattern used in the deterministic harness.

The three biases that quietly wreck judge scores

An LLM judge is not a neutral instrument. The 2024 arxiv survey A Survey on LLM-as-a-Judge catalogs a dozen failure modes; three of them cause most of the damage in practice.

Position bias

In pairwise mode, models systematically prefer whichever answer is shown first (or sometimes second). If you only ever show candidate A first, your win rates are inflated fiction.

The fix is to grade every pair twice, swapping the order, and only count a win when both orderings agree. Disagreement becomes a tie.

PAIRWISE = """Compare two answers to the QUESTION. Decide which better satisfies it
on correctness and relevance only. Ignore length and style.
Return ONLY JSON: {"winner": "A" | "B" | "tie", "reason": ""}."""

def _one_pair(question: str, a: str, b: str) -> str:
    msg = client.messages.create(
        model=JUDGE_MODEL, max_tokens=200, temperature=0, system=PAIRWISE,
        messages=[{"role": "user",
                   "content": f"QUESTION:\n{question}\n\nANSWER A:\n{a}\n\nANSWER B:\n{b}"}],
    )
    return json.loads(msg.content[0].text.strip())["winner"]

def judge_pairwise(question: str, first: str, second: str) -> str:
    # Grade both orderings to cancel position bias.
    fwd = _one_pair(question, first, second)   # first shown as A
    rev = _one_pair(question, second, first)   # first shown as B
    if fwd == "A" and rev == "B":
        return "first"
    if fwd == "B" and rev == "A":
        return "second"
    return "tie"   # judge flipped with order -> not a real preference

Verbosity bias

Judges tend to score longer answers higher, even when the extra words add nothing. Two defenses: state "do not reward length" in the rubric (we did), and spot-check by feeding the judge a padded copy of a good answer. If the score jumps, your rubric is leaking verbosity preference and needs tightening.

Self-preference bias

Models rate their own outputs more favorably than a neutral grader would. If the agent under test is Claude, do not treat a Claude judge as fully independent. Mitigations, in order of strength: use a judge from a different model family, use a different tier than the one being tested, and always calibrate the judge against a small set of human labels before you trust its absolute numbers. The model-graded eval templates in the OpenAI Cookbook are a useful cross-family second opinion when self-preference is a real concern.

Calibrate before you trust the number

A judge you have not calibrated is a random number generator with good grammar. Hand-label 15 to 30 examples, run the judge, and measure agreement. Even a rough exact-match rate tells you whether the rubric is usable.

def calibrate(labeled: list[dict]) -> float:
    """labeled = [{'question':..., 'answer':..., 'human': <1-5 int>}, ...]"""
    hits = 0
    for row in labeled:
        verdict = judge_pointwise(row["question"], row["answer"])
        if verdict["score"] == row["human"]:
            hits += 1
    return hits / len(labeled)

# agreement < ~0.6 on a 5-point scale -> tighten the rubric anchors or drop to pass/fail

If agreement is poor, the usual cause is a vague rubric. Collapse the 5-point scale to pass/fail, sharpen the anchors, and re-measure. A judge that agrees with humans on a binary call is worth far more than one that produces precise-looking 1-to-5 scores nobody can reproduce.

Verify your setup (break it on purpose)

The signature check: prove the harness catches the biases it claims to. Run these three and confirm the expected behavior before wiring the judge into CI.

1. Identical answers, pairwise. Pass the same string as both first and second. A well-behaved judge must return "tie". If it declares a winner, position bias is live and your swap logic is the only thing saving you.
2. Verbose-but-wrong vs terse-but-right, pointwise. Grade a padded, confident, incorrect answer against a short correct one. The correct one must score higher. If not, your rubric leaks verbosity preference.
3. Empty answer. judge_pointwise(q, "") must score 1. A judge that scores an empty string above 1 will happily pass silent agent failures.

q = "What year did the Apollo 11 crew land on the Moon?"
assert judge_pairwise(q, "1969.", "1969.") == "tie"
assert judge_pointwise(q, "It was 1969.")["score"] > \
       judge_pointwise(q, "Historians and enthusiasts alike have long debated the "
                          "storied timeline of lunar exploration, and it was 1968.")["score"]
assert judge_pointwise(q, "")["score"] == 1
print("judge harness verified")

If any assertion fails, fix the rubric, not the test. The whole point of a judge is that it fails loudly on the cases you designed it to catch.

Limitations and open questions

• A judge measures agreement with your rubric, not ground truth. Calibration bounds how far you can trust it; it does not turn subjective scores into facts.
• Absolute pointwise scores stay noisy. For release gates, prefer pairwise win rate against a frozen baseline over "did the average score cross 4.0".
• Cost scales linearly. Pairwise-with-swap is four judge calls per comparison (two orderings). On large datasets, sample rather than grade every item, and reserve the judge for the slices deterministic checks cannot cover.
• Open question: how much does judge-model choice change your rankings? A worthwhile follow-up is to run the same eval set through two judge families and report where their verdicts diverge, the kind of reproducible, documented comparison that survives a model upgrade.

Start deterministic, add a calibrated judge only where you must, and verify the judge catches its own biases before you trust a single score.