What You Need to Know
An AI-assisted developer workflow hands part of an engineer's work to an AI tool. An outcome measure is a property of the work itself. An adoption figure counts how much the tooling was used. A check tells you cheaply whether an output is right. Packaging turns one person's procedure into a shared team artefact.
Take a laundrette chain whose two developers maintain its booking app and payment-machine software. A quarter after adopting AI coding tools, the owner asks whether they worked.
Measure the Work, Not the Tooling
Outcome measures are properties of the work:
- cycle time from ticket to merged change
- defect escape rate
- review latency
- how long a new joiner takes to make a safe change
- incident time-to-resolution
Each has a before and moves for reasons an architect can defend in a review.
Adoption figures measure how much the tooling was used: sessions per week, accept rate, the share of merged changes containing assisted code. Anthropic's analytics guidance says to read them alongside engineering measures, never in place of them. A productivity claim with no baseline fails like an accuracy claim without one.
Name the outcome measure and its baseline before the quarter starts. A baseline reconstructed afterwards is an argument, not evidence. At the laundrette, daily tool use is an adoption figure; payment bugs reaching the machines, counted last quarter too, is an outcome measure.
Verifiability Decides What to Automate
Before automating a task, ask what checks its output cheaply. That question sorts the work. A test suite, build result, linter or fixture comparison closes the loop without a human inside it. The agent works, runs the check, reads the result and iterates. Without a check it can run, Anthropic warns, "you become the verification loop: every mistake waits for you to notice it".
Work whose correctness only an experienced human can judge, slowly, is the worst candidate for automation, not the best. Volume arrives faster than judgement absorbs it, so automation makes the review bottleneck bigger.
At the laundrette, an agent can change the payment code and run its test suite. Judging whether a pricing rule is fair needs the owner, so it comes last.
The Task That Hurts Most Is Not the Task to Automate First
The tempting option points automation at the painful, unverifiable work. The keyed answer starts where the check is fast and objective.
Packaging Turns a Habit Into a Capability
One engineer's good procedure is a habit; packaging converts it into the artefact a team needs. The reuse unit follows what is being reused:
- a repeatable procedure becomes a shared, version-controlled workflow anyone can invoke by name
- a recurring specialist task becomes a defined role with its own instructions and restricted tool access
- a recurring context need becomes shared project context, not four private copies
Outcome quality then stops depending on who ran it. Improving the artefact once improves it for everyone, because it is reviewed and versioned like code. The gain also survives a resignation; a habit does not. At the laundrette, one developer's release-check routine, committed as a shared workflow, still runs if she leaves.
Review Stays Independent, Humans Keep Merge Authority
Review is where quality is enforced, and its value collapses when the reviewer shares the author's context. Whatever produced a change still holds the assumptions it was written under. So it confirms its own reasoning back, fluently and at length. Anthropic states the fix: "A fresh context improves code review since Claude won't be biased toward code it just wrote."
Humans keep merge authority. Anthropic's managed review posts findings that "don't approve or block your PR", so machine review adds a signal without taking the decision. An independent reviewer has its own failure mode: asked to find gaps, it reports some even when the work is sound. Scope it to correctness and stated requirements, not taste. At the laundrette, a fresh context reviews the tool's changes for correctness; a named developer decides what merges.
Autonomy Follows Reversibility
Autonomy tracks how cheaply an action can be undone, not how much the team trusts the tooling. A mistake that reverts cheaply costs little; one that cannot needs a person first. So:
- bound what automation may execute
- gate irreversible actions behind an approval
- protect the branches that matter
- let continuous integration be the objective gate that no confident narration talks its way past
At the laundrette, payment-machine changes wait for the owner's approval, and continuous integration blocks merges that fail the payment tests.
Adoption follows the same discipline. Start where the pain is measurable, make the gain inheritable (objective 7.1), and let evidence spread it, not a mandate.
In the exam
Items in this objective sit inside one question: did the tooling work? They test what counts as an answer, which work should have been automated at all, what turns one engineer's habit into a team capability, and who signs the result off. Little turns on a product feature. Most turns on telling a usage number from a result.
The usual trap automates the painful work with no check. Packaging is the easily missed half of most keyed answers.
Caveats
A same-period comparison shows whether the numbers moved, not why. Fewer laundrette payment bugs would not prove the tooling caused the fall.
Exam Traps
Reporting an adoption percentage, an accept rate or a satisfaction survey as the return on the investment
Each counts how much the tooling was used or how it felt, not what changed in the work. With no baseline on cycle time, defect escape rate or review latency, the figure cannot separate a genuine gain from busier tooling.
Letting whatever produced a change review it, then recording the change as reviewed
The reviewer still holds the assumptions the change was written under and will confirm its own reasoning back fluently. Review belongs in a fresh context or with a different reviewer, and a named human keeps merge authority.
The absolutes: mandate adoption, standardise on the pilot's choices before anything is measured, adopt every new tool on release, or forbid developers from editing generated code
Each swaps judgement for a rule. A mandate buys compliance, premature standardisation generalises from an unmeasured pilot, continuous adoption produces churn, and forbidding edits keeps the risks while discarding the tool.
Measuring the improvement honestly but leaving the procedures that produced it in individuals' heads
This is the second-best option the exam plants for near-passers. An unpackaged procedure cannot be reviewed, cannot be improved once for everyone, and walks out of the door with whoever invented it.
Practice Scenario
A platform team is a quarter into AI-assisted tooling and the director asks whether it worked. The only figure anyone holds is that 70% of merged changes contained assisted code. Two engineers have worked out procedures that clearly help — one for generating migration tests, one for triaging production logs — and neither has been written down. What should the architect do?
Build Exercise
Build the Workflow Inventory and Its Outcome Measure
What you'll learn
- How to separate outcome measures from adoption statistics
- How to choose the reuse unit for a procedure that is still a habit
- How to set a baseline before the comparison rather than after it
- How to keep review independent of whatever produced the change
- How to size autonomy to the reversibility of the action
- List every repeatable procedure your team runs with AI assistance, and mark each one as packaged and shared, or still a habit that lives with one person
Why: The inventory is the only way to see how much team capability is currently held by individuals.
You should see: One row per procedure, with the unpackaged rows outnumbering the packaged ones on a first honest pass.
- For each unpackaged procedure, name the reuse unit it should become - a shared invokable workflow, a defined role with its own instructions and restricted tools, or shared project context - and say who inherits it and where it is committed
Why: The reuse unit follows what is being reused, and inheritance is what makes the gain a team asset rather than a personal one.
You should see: Every unpackaged row assigned a unit and a home that a new joiner receives on clone.
- Choose one outcome measure per procedure, name the baseline you would compare it against, and name one adoption statistic you will deliberately not report as a result
Why: Naming the statistic you refuse to present is the fastest way to find out whether you hold a real measure.
You should see: A measure and a baseline per procedure, plus one adoption figure explicitly ruled out.
- Write the review rule: what reviews the changes the tooling produced, in which context, what it is scoped to look for, and who holds merge authority
Why: Review is the load-bearing control, and it stops working when the reviewer shares the assumptions of whatever wrote the change.
You should see: One paragraph naming an independent reviewer, a narrow scope, and a named human with merge authority.
- Audit the leash. For every automated or unattended run, record what it may execute, what is gated behind an approval, what continuous integration blocks regardless, and which of those rest on a written instruction rather than an enforced control
Why: Autonomy should track reversibility, and the rows resting on instruction are the findings.
You should see: A short table per automated run, with an honest count of the controls that are guidance today.
Sources
- Best practices for Claude Code — Anthropic
- Track team usage with analytics — Anthropic
- Extend Claude with skills — Anthropic
- Code Review — Anthropic
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 7 — Anthropic