Skip to content
Domain 5
14%

Glossary: Governance, Safety & Risk Management

Quick-lookup definitions for the 14% exam domain. Each entry includes a concise definition and exam context. Follow the lesson links to dive deeper.

Control Surface

One of the five fixed positions in the request path where a control can sit: input screening, the instruction layer, capability scoping, output validation, and architectural removal. Items are built by naming a risk and offering one option from several surfaces, so the skill is reading which surface the stated risk actually needs.

Exam context: Every option in a well-written stem is a real control. Read the stem for the phrase that fixes the surface — never reaches, before it is sent, cannot be undone, after the fact — and the elimination becomes mechanical.

See also: 5.1 Guardrails, Safety Controls, and the Surface a Risk Belongs To


Architectural Removal

Declining to build a capability at all, rather than building it and guarding it. The strongest answer available, because a capability that does not exist cannot be misconfigured, cannot decay, and costs nothing to audit.

Exam context: The guide's Domain 3 sample keys removal over logging, a confirmation prompt and a bigger model. Where a stem says the role never needs the capability, options adding friction to it are real controls answering a question nobody asked.

See also: 5.1 Guardrails, Safety Controls, and the Surface a Risk Belongs To


Instruction-Layer Control

The system prompt's scope, refusal policy and escape hatch. It shapes behaviour and earns its place, but it is context rather than enforcement: if the capability is wired, an instruction not to use it is a request.

Exam context: The highest-frequency wrong answer in the domain, and it recurs in Domains 3 and 7. Run one test on any stem — what stops a confused or compromised turn from reaching the wired capability? "The prompt says not to" leaves the risk unowned.

See also: 5.1 Guardrails, Safety Controls, and the Surface a Risk Belongs To


Output Gate

Validation applied to a response before it leaves: schema and policy checks, blocked claim classes such as financial commitments or clinical advice, required citations, personal-data redaction on egress. It blocks, redacts, or routes the response to a person.

Exam context: A requirement written as a guarantee has only two homes, the output gate or removal. A gate that only records what it saw is a detective control wearing a preventive label, which is how the logging distractor is usually dressed.

See also: 5.1 Guardrails, Safety Controls, and the Surface a Risk Belongs To


Untrusted Content

Everything the system did not author: retrieved documents, tool results, connector payloads, uploaded files, web content. It is delimited and reasoned about as data, never followed as instructions, and it arrives inside tool results with its origin declared and its encoding unambiguous.

Exam context: The canonical security item of the domain. Direct jailbreaks make the user the adversary; indirect injection keeps the user trusted and rides the instruction in on third-party content. Screening reduces frequency, scoping bounds consequence.

See also: 5.1 Guardrails, Safety Controls, and the Surface a Risk Belongs To


Defence in Depth

Independent layers arranged so that one failing does not put harm in front of a user, which is why a well-written stem can carry two good-looking options. Depth is not permission to place a consequence on a layer that cannot hold it.

Exam context: An irreversible external action is governed by capability scoping or a human gate, never by firmer wording plus a weekly sample. Ask whether one layer is being asked to carry a consequence it cannot.

See also: 5.1 Guardrails, Safety Controls, and the Surface a Risk Belongs To


Risk Register

An architecture artefact rather than a document. Five fields per row: the mechanism, the blast radius, the detection signal, the owning control with its control surface, and the residual with the name of whoever accepted it.

Exam context: A risk with no owning control is an accepted risk, and acceptance is a decision somebody signs. A list with no controls, owners or signals discharges nothing, and the keyed answer is usually the option that produces one.

See also: 5.2 Risks, Limitations, and the Failure Modes That Stay Silent


Blast Radius

How far a consequence reaches, who bears it, and whether it can be undone. For an injection row it equals the agent's permission set, which is why the row names the permissions rather than the technique: the technique will change, the permission set is what you control.

Exam context: Reversibility and exposure, not frequency, are what a stem hands you to reason with. Ranking by likelihood alone puts a frequent reversible failure above a rare irreversible one.

See also: 5.2 Risks, Limitations, and the Failure Modes That Stay Silent


Detection Signal

The required "how would we notice" column: what moves in the telemetry while a failure is happening, named as a specific metric, log or alert. Any row whose honest answer is that a customer would complain has no detection at all.

Exam context: Complaint-driven detection fires after the harm and catches only the harms people bother to report, which is a biased sample. The risks that hurt most fail silently in paths nobody instrumented.

See also: 5.2 Risks, Limitations, and the Failure Modes That Stay Silent


Confident Fabrication

The characteristic failure of these systems: fluent, plausible, unflagged output that never appears in an error rate because nothing errored. Anthropic's own guidance closes on the point the register depends on, that its techniques reduce hallucination substantially without eliminating it.

Exam context: Anything consequential therefore needs a verifiable basis rather than a plausible one. "We tested it thoroughly and it worked" is offered as a risk assessment and is a measurement of one axis under one distribution.

See also: 5.2 Risks, Limitations, and the Failure Modes That Stay Silent


Compounding Error

The way a modest per-step error rate accumulates across an agent loop, with each additional autonomous action widening the blast radius. Anthropic states it directly: autonomy brings higher costs and the potential for compounding errors, and recommends extensive sandboxed testing with guardrails.

Exam context: The levers are scope, step count, reversibility and checkpoints. A per-item accuracy figure quoted for a multi-step autonomous workflow has told you almost nothing, because it carries none of them.

See also: 5.2 Risks, Limitations, and the Failure Modes That Stay Silent


Third-Party Surface

Every connector, MCP server, subprocessor, analytics vendor and observability platform in the path: parties that can see data and sometimes act. Anthropic's retention documentation places third-party integrations outside its arrangements entirely and tells customers to review each service's data handling.

Exam context: Their access is your risk posture and their agreements are your compliance posture, which is where the register hands off to regulatory translation. The vendor nobody assessed is the row most often missing.

See also: 5.2 Risks, Limitations, and the Failure Modes That Stay Silent


Placement Principle

The rule that decides where a human gate goes: the consequence is high and the action is hard to reverse. Gate everything and the system delivers nothing anyone will keep paying for; gate nothing and irreversible harm is left to a probability.

Exam context: Reversibility and blast radius decide, not seniority, squeamishness or a fixed percentage. Where a stem supplies both a throughput constraint and a reversibility distinction, the keyed answer splits the flow rather than applying one pattern uniformly.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Approve-Before-Execute

The pattern where the system proposes, a human authorises, and only then does the action fire. It belongs on irreversible external effects: payments, deletions, outbound customer messages, production changes, anything with a legal or financial consequence.

Exam context: Applied to the reversible bulk it becomes the approve-everything distractor, which fails the stated business requirement and manufactures the rubber-stamping the gate was meant to prevent.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Review-Before-Release

The pattern where the system drafts and a human signs off, so the output ships under that person's accountability. It belongs on consequential content: regulated advice, clinical documentation, statements that commit the organisation.

Exam context: In the High-risk Areas named by Anthropic's Usage Policy the qualified reviewer is a policy requirement rather than a design option, so options substituting monitoring or a caveat for that person are wrong before the judgement is weighed. The requirement attaches to a recommendation about one person, either advice they might rely on or a decision about them, and the policy lists exclusions: wellness content on sleep, stress, nutrition and exercise is one. Where policy does not compel the reviewer, the pattern is still often right on consequence alone, which is the judgement the item is testing.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Triggered Escalation

The pattern where the system handles the routine and routes the uncertain, out-of-policy or high-value case to a person. The whole pattern rests on the trigger, which is validated against outcomes like any other classifier.

Exam context: An unvalidated confidence threshold is a lottery dressed as a control: a classifier with unknown error rates deciding which cases a human ever sees, whose misses are invisible by construction.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Post-Hoc Sampled Audit

The pattern where nothing is gated and a sample keeps the quality claim honest. Right for high-volume, reversible, low-severity work, and it is also what stops the other patterns decaying, because it is how you discover that reviewers stopped reading.

Exam context: Placed on an irreversible action it becomes the retrospective-review distractor. A sample keeps a claim honest; it does not recall a payment, a deletion or a message that has already reached a customer.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Reversibility Guarantee

Automating with no human gate, having engineered the action to be cheap to undo: an audit trail, an undo path, a compensating transaction. A legitimate keyed answer wherever the stem establishes that the action is low-impact and genuinely reversible.

Exam context: Candidates trained to reach for a gate miss this option. The failure mode is a guarantee that was assumed rather than built, and the window matters: reversible until a batch closes is not reversible afterwards.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Meaningful Human Review

A gate the reviewer can actually act on. The ICO's test is that a decision does not fall outside the automated-decision rules just because a human rubber-stamped it: reviewers need the authority and competence to go against the recommendation, and a reviewer restricted to the data the system used is arguably weighing no additional factors at all.

Exam context: The reviewer needs the trace, the sources, the flagged uncertainty, at least one input the model never had, and enough time per item. The measurement is the proof: override rate, reviewer agreement, time per item.

See also: 5.3 Human-in-the-Loop Validation: Placement, Pattern, and Proof


Full-Path Scope

Drawing the compliance boundary around every hop the regulated data reaches rather than around the application: the model provider, the retrieval index and its embeddings, prompt and response logs, observability traces, evaluation datasets, analytics vendors and their subprocessors.

Exam context: Traces and eval sets are the forgotten copies, built deliberately by Domains 3 and 4. A stem naming an observability vendor or an evaluation dataset is testing this, and options that fix the application while leaving the copies alone are planted.

See also: 5.4 Regulatory Compliance as an Architecture Constraint


Minimisation at the Boundary

Sending the fields the task needs and no more, with redaction, tokenisation or pseudonymisation applied before regulated data crosses a trust boundary and the re-identification key kept inside it. The ICO extends it to derived datasets, conditionally: depending on the likelihood and severity of the risk to individuals, de-identification techniques may also be needed on training data before it is extracted from its source and shared internally or externally.

Exam context: In an over-broad-flow stem the keyed answer reduces what crosses. Sort the options by what still crosses afterwards, and only one of them changes that number.

See also: 5.4 Regulatory Compliance as an Architecture Constraint


Business Associate Contract

The HIPAA agreement that has to exist before protected health information moves. It establishes permitted uses, requires safeguards and breach reporting, binds subcontractors to the same restrictions, supports access, amendment and an accounting of disclosures, and at termination requires the information returned or destroyed with no copies retained where that is feasible, extending the contract's protections to anything that cannot be.

Exam context: Flow-down and return-or-destroy are the clauses that break naive architectures, because they reach every vendor in the path and every copy in the eval set. "We will paper it at the next renewal" is always wrong once data is already flowing.

See also: 5.4 Regulatory Compliance as an Architecture Constraint


Authorisation Boundary

All components of an information system to be authorised for operation by an authorising official, excluding separately authorised systems it connects to. The definition comes from OMB Circular A-130 and NIST SP 800-37, not from the FedRAMP programme, which is what makes it the durable half of the topic.

Exam context: A FedRAMP authorisation certifies that a service completed the process; it is not an authorisation to operate, which the agency's own official still grants. An unauthorised component carrying federal information drags its assessment burden inside the agency's boundary.

See also: 5.4 Regulatory Compliance as an Architecture Constraint


The Deletion Story

The engineering answer to a data-subject erasure request: the list of every copy you would have to find, and the honest verdict on which you currently could not. Access, correction, deletion and portability all depend on provenance and an inventory.

Exam context: Vector indexes, trace stores and evaluation fixtures are where deletion requests go to die. A provider's retention or residency arrangement governs that provider, never what your own copies do with the same records.

See also: 5.4 Regulatory Compliance as an Architecture Constraint


Fairness Through Unawareness

The false comfort of dropping a protected attribute from the inputs and declaring the system fair. The ICO names the failure directly: features correlate with protected characteristics in non-obvious ways, and modern machine learning finds those combinations and redundant encodings more readily than traditional statistics.

Exam context: The highest-frequency wrong answer in the domain, and it is written to look rigorous. Knowing why exclusion fails, which is proxies, is the reasoning being tested.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency


Proxy Variable

An ordinary feature that carries the signal of a protected characteristic: working hours, occupation, application channel, address history, device, language register. The ICO's worked case is a redundancy ranking on hours worked discriminating by sex where more women in a sector work part-time.

Exam context: Learn the shape rather than one signature example. Proxies are also only one entry point: bias enters through measurement and labelling, through the builders' assumptions, through an objective that embeds who is worth optimising for, and through an inaccessible interface.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency


Disaggregation

Splitting outcome measurements across cohorts instead of reporting an aggregate. Aggregate accuracy is a weighted average dominated by the majority population, so a cohort failure stays invisible in it at any level of statistical precision.

Exam context: Cohorts come from a harm hypothesis — who could this disadvantage, along which axis — rather than from whichever field is available. A tighter confidence interval on the aggregate offered instead of a breakdown is a planted answer.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency


Fairness Criterion

The stated, documented choice among criteria that cannot generally all hold at once: equal error rates across groups, equal selection rates, calibration within groups. Fairness is therefore a decision tied to the harm the system can do, not a property a system has or lacks.

Exam context: The keyed answer names a criterion and justifies it against the decision being made. The ICO adds the limit: fairness metrics identify discriminatory impacts but are not likely to guarantee compliance or explain why the unfairness occurs.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency


Remediation Ladder

The ordering of fixes, cheapest and most durable first: fix the data, fix the framing, constrain the decision, then monitor continuously because the population drifts and a one-off audit ages.

Exam context: Two options are not rungs. Hiding a score removes the evidence rather than the harm, and recording the base model's training data as root cause is fatalism dressed as rigour, because framing, constraint, remit and monitoring all belong to the application team.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency


Point-of-Use Disclosure

Telling the person, where and when they are using the system, that they are dealing with an AI rather than a human, what its limits are, and how to reach a person. For consumer-facing chatbots and AI agents that talk directly to external users, Anthropic's Usage Policy sets the baseline: at a minimum at the beginning of each chat session or in the product interface.

Exam context: A model card, a policy page or a paragraph in the terms answers a different question for a different audience. Deliberately keeping the system's nature ambiguous to preserve trust is the trap that looks commercially sensible.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency


Outcome-Based Explanation

The reasoning behind this person's decision, in plain everyday language, saying whether a human was meaningfully involved, with a timely route to contest it. The ICO separates it from a process-based explanation, which describes the governance the organisation followed.

Exam context: Read which of the three transparency obligations the stem is asking about, because distractors answer one of the other two competently. A description of the architecture is not an explanation, and "the model determined" is not a basis. Accountability travels with it: automating a decision does not move it onto the tool.

See also: 5.5 Ethical AI: Bias, Fairness, and Transparency