What You Need to Know
An evaluation dataset is the set of cases a system is tested against. A golden set is a dataset trusted for comparing scores over time. A grading method scores each case: a code-graded check, an LLM judge, or human review. This objective builds the machinery that measures objective 4.1's metric sheet.
A physiotherapy clinic uses Claude to turn session notes into home-exercise plans. Its metric sheet asks for plans in the clinic's format, faithful to the notes, and safe from tampering.
Four Case Populations
Dataset composition is risk coverage: each population protects against a different failure. Hand-written happy-path cases certify only the demo; they never reach the inputs that break things.
- Representative cases mirror the production distribution and certify the everyday path. They are the bulk of the set. For the clinic, these are routine notes from knee, back and shoulder appointments.
- Edge cases sit at the boundaries where behaviour changes. Anthropic's evaluation guidance names the populations people forget: irrelevant data, overly long inputs, poor user input, and ambiguous cases. The clinic adds a note padded with a parking complaint, a very long note, a note in shorthand, and a note that says only "progress as tolerated".
- Adversarial cases operationalise the safety and security metrics from 4.1: injection attempts, leading questions, malformed input. Anthropic's guardrails guidance says to test your workflow with content that deliberately contains injection attempts before deploying. Security is measured under attack. Without attack cases, it is only asserted. The clinic adds a note that tells Claude to ignore the plan template.
- Regression cases are minted from production failures, so no fixed bug returns unnoticed. A bug with no regression case is a bug you pay for twice. When one plan named the wrong knee, that note became a permanent case.
Golden-Set Discipline
A golden set earns trust through three properties.
- Versioned. A score is comparable to last month's only if the set is the same or the change is recorded. Unversioned sets make every trend line meaningless. The clinic tags each revision of its set.
- Held out. A case that appears in the prompt as a few-shot example is no longer a test. Claude has already seen it, so passing proves recall, not capability. Held out means held out of the prompt, not just out of the training data. No example plan in the clinic's prompt may appear in its set.
- Refreshed. Production traffic drifts. A static launch set slowly certifies a distribution that no longer arrives. Refresh cadence is a design decision to make on purpose. When the clinic starts taking post-surgery patients, their notes join the set.
Sizing by Risk and Variance
LLM outputs are non-deterministic: the same note can give a different plan each run, so small samples mislead. Anthropic's research on statistical model evaluations sets out practices including:
- Report the standard error of the mean and confidence intervals rather than bare scores.
- Use clustered standard errors when questions are related, because naive errors can understate uncertainty substantially. Several notes from one patient are related cases.
- Resample non-deterministic answers and average per question.
- Run a power analysis to choose the sample size before running the eval.
Sizing to a round number is folklore. High-stakes or high-variance behaviours need more cases before a result deserves trust. The clinic gives post-surgery plans more cases than routine back pain.
Mixed Methodologies: The Pairing Rule
The pairing rule: deterministic where possible, judge where not, humans to calibrate the judge and audit the tail.
- Code-graded checks (exact match, schema validation, regex, programmatic constraints) are cheap, fast and objective. Use them for everything they can grade. Anthropic's design principles point the same way: automate when possible, and prioritise volume. More questions with slightly lower-signal automated grading beat fewer hand-graded ones. The clinic checks every plan's format with schema validation.
- LLM-as-judge scores open-ended quality, such as tone, completeness and faithfulness, at scale. Anthropic positions LLM-based grading for subjective judgements that traditional metrics cannot quantify. It builds its example judges with explicit scales and definitions. Anthropic calls it best practice to use a different model to evaluate than the one that generated the output. The rubric has explicit scoring anchors and is versioned like code. The clinic's judge scores whether each plan is faithful to the notes.
- Human review is the gold standard. It is too expensive to grade everything and too important to skip. Spend it calibrating the judge, auditing samples, and adjudicating high-stakes or ambiguous cases. At the clinic, a senior physiotherapist labels a sample of plans.
An Uncalibrated Judge Is Not Evidence
The judge is itself a model with failure modes, scoring a system it resembles. Standard practice is to validate it against human labels on a sample before trusting its scores. Until then, its numbers carry no evidential weight, however precise they look.
In the exam
The exam tests two design skills: composing a dataset that covers the system's real risks, and pairing each metric with a grading method that can honestly score it. "Mixed methodologies" in the blueprint means code-graded checks, LLM judges and human review, each assigned the job it does best. The exam rewards the pairing rule. When a stem describes recurring production incidents alongside a green eval suite, the missing populations are usually the answer. Know that the sizing machinery exists.
The three golden-set properties are standard evaluation discipline, not Claude-specific machinery.
Exam Traps
Trusting an LLM judge that nobody has checked against human labels
An unvalidated grader is a model with unknown failure modes scoring a system it resembles. Calibration against a human-labelled sample is what converts its scores from decoration into evidence.
Choosing full human grading as the rigorous option
It mis-spends the gold standard. Anthropic's guidance is to prioritise volume with automated grading; humans calibrate the judge and audit the ambiguous tail. All-human grading buys less coverage at more cost.
Pasting eval cases into the prompt as few-shot examples, then celebrating the pass rate
The eval now measures memory, not capability. Held out means held out of the prompt — leakage is the quietest way an eval suite goes green while the system decays.
Growing the dataset to a round number and calling it rigorous
Size follows risk and variance, not aesthetics. Anthropic's statistical evals research prescribes power analysis and standard errors — a big set sized by feel can still be too small where variance is high and wastefully large where it is not.
Practice Scenario
A team evaluates a customer-email assistant with 40 hand-written happy-path cases graded by an LLM judge nobody has calibrated. Production incidents keep recurring. What should change?
Build Exercise
Design the Dataset Composition for One System
What you'll learn
- How to compose the four case populations and defend what each protects against
- How to version a golden set and record regression-case provenance
- How to detect few-shot leakage before it invalidates the suite
- How to assign each metric a grading method using the pairing rule
- How to write a judge rubric and plan its calibration against human labels
- For a Claude system you know, write the four populations with counts: representative, edge, adversarial, regression — and one sentence on what each population is protecting against
Why: Composition is risk coverage. Making the protection explicit exposes the risks the current set silently ignores.
You should see: Four population entries, each with a count and a named risk — not a single undifferentiated case list.
- Record the provenance of every regression case (which incident minted it) and the version tag of the current set
Why: Provenance keeps regression cases honest, and versioning is what makes scores comparable across time.
You should see: Each regression case linked to an incident, and a version identifier the next eval run will cite.
- Check for leakage: list every few-shot example in the prompt and confirm none appears in the eval set
Why: A leaked case measures memory, not capability, and silently inflates every score the suite reports.
You should see: A leakage check with an explicit result — ideally an automated comparison, not a visual skim.
- Assign each metric from your 4.1 metric sheet a grading method: code-graded, LLM judge with rubric, or human review — and justify each assignment with the pairing rule
Why: The pairing rule is the skill this objective examines: deterministic where possible, judge where not, humans to calibrate and audit.
You should see: Every metric matched to exactly one primary grader, with a one-line justification.
- For the judge: write the rubric with explicit scoring anchors, version it, and plan the calibration sample of human-labelled cases it must agree with before its scores are trusted
Why: The rubric is the judge specification, and calibration is what gives its scores evidential weight.
You should see: A rubric with a defined scale and anchor descriptions per score, a version tag, and a calibration plan with an agreement threshold.