What You Need to Know
An optimisation cuts a Claude system's tokens, latency or cost. A lever is one such change. The dominant term is the biggest part of the bill or the wait. Every lever is good somewhere. The skill is matching one to the workload's dominant term within the 4.1 quality bar.
Take a restaurant whose owner wants a smaller Claude bill without worse answers. It runs a booking chat, an invoice pipeline and a nightly review sort.
Rule One: Measure First
Traces and usage data name the dominant term: stage, token type, call pattern. Anthropic's Usage and Cost Admin API gives programmatic breakdowns of usage by model, workspace and API key, and of cost by workspace. Documented uses include cache-efficiency analysis and budget alerting. If most spend is input tokens on one stage, tuning output tokens elsewhere is no improvement. Optimising the most visible line instead is an easy mistake to miss. Trimming the restaurant's chat replies saves nothing while invoice input dominates.
Rule Two: Re-gate With Evals
The quality bar is a constraint. After each change, such as a cheaper invoice model, the evals run again. The change ships only if it is cheaper and still clears the bar. Shipping without the rerun is a regression: the saving is measured, the quality loss is not. Anthropic's effort guidance agrees: step down effort where evals show quality holds.
Token and Cost Levers
- Prompt caching suits repeated stable prefixes: long system prompts, policy documents, tool definitions sent on every request. Cache reads are billed at a small fraction of the base input price, writes at a premium over it (current figures are in the Domain 4 quick reference). The cached content must be a byte-stable prefix, so stable content goes first. Domain 2 owns the mechanics. The booking chat sends the same allergen policy and tool definitions every time, so it has a prefix to cache. Select caching only when the workload has a repeated prefix.
- Context curation sends the fields and leaves the rest of the file out. The cheapest token is one that never enters the request. The invoice stage sends the supplier, line items and totals.
- Tight output contracts cap output tokens, which are the expensive ones. The invoice stage returns a fixed set of fields.
- Effort and reasoning budgets tune spend to the task within a single model: low effort for simple classification and high-volume work, high effort for complex reasoning. Anthropic states that tuning effort is often a better lever than switching models. Thinking spend on a trivial task is waste. Sorting reviews into praise and complaints is simple classification, so it runs at low effort.
- Right-sizing per stage gives each pipeline stage the cheapest model that clears its measured bar. An escalation cascade routes the hard tail to a stronger tier. This is Domain 2's pattern, and it beats both premium everywhere and blind downsizing. Most invoices go to a small model; the hard ones escalate.
- Batch processing runs work that is not time-sensitive asynchronously at a discount, with documented turnaround and expiry windows (figures in the Domain 4 quick reference). Anthropic positions it for large-scale evaluations, content moderation and bulk operations that do not need immediate responses. The cue is the absence of a waiting user: "overnight", "bulk" and "no one is waiting" point to batch, and an interactive surface rules it out. Nobody waits for the nightly review sort, so it can go to batch. The booking chat cannot.
Latency Levers
Match the lever to the latency the consumer feels: time-to-first-token at an interactive surface, total completion time in a pipeline (the 4.1 distinction).
- Streaming attacks perceived latency. Users see output as it generates, and Anthropic recommends it for responsive interactive experiences. Total completion time is unchanged, so it is the right lever only where a human is watching. It suits the booking chat and does nothing for the invoice pipeline.
- The smallest model that clears the measured bar attacks real latency. It is the first recommendation in Anthropic's latency guidance.
- Parallelising independent tool calls removes serial waits. If the booking chat checks table availability and opening hours as two separate calls, it can make both at once.
- Caching improves time-to-first-token for long stable prefixes, as well as cost.
Optimisations Move Along the Frontier — They Do Not Repeal It
Every lever spends something to save something. Streaming spends nothing but changes only perception. Batch spends turnaround time. Downsizing spends capability headroom. Caching spends nothing only when the prefix is genuinely stable. Name the spend and the saving, with eval evidence the spend is affordable.
In the exam
Professional-level optimisation questions match a lever to the stem's dominant term. Two rules bracket the objective, and most distractors skip one. In cost-versus-quality stems, right-sizing with a cascade is the balanced answer. An option promising savings with no named cost and no re-gate is the planted distractor.
Exam Traps
Offering batch processing's discount for a latency-sensitive, user-facing flow
Batch is asynchronous — its cue is the absence of a waiting user. An interactive surface cannot wait for a discounted answer, however large the saving. Read the stem for who is waiting.
Switching to the smallest model as a cost fix without re-running the evals
Domain 2's trap recurring as the optimisation that skipped its evidence: the saving is provable, the quality loss is unmeasured, and the bar is a constraint, not a preference. Right-sizing is eval-gated by definition.
Offering prompt caching for context-window overflow, wrong answers, or unique one-off documents
Caching reuses a stable repeated prefix to cut cost and time-to-first-token. It does not shrink the context, add capacity, or change what the model says — and a workload of unique documents has no repeated prefix to reuse.
Practice Scenario
A nightly compliance sweep summarises 30,000 different documents on a premium model with per-request calls; finance flags the cost and wants it cut by changing the pipeline itself. Evals show a mid-tier model matches summary quality on a representative sample, and no model may replace the premium one unless tests show it meets that bar. What should the architect do?
Build Exercise
Run a Measured Optimisation Pass on One System
What you'll learn
- How to identify the dominant cost and latency terms from traces and usage data
- How to match levers to dominant terms and reject levers that do not move them
- How to state what each optimisation spends to save what
- How to define the eval re-gate an optimisation must pass before shipping
- How to project savings at volume against a measured baseline
- Pull a week of traces and usage data for one Claude system and name the dominant cost term and the dominant latency term (stage, token type, call pattern)
Why: Without measurement the effort lands on the most visible line, not the dominant term — and the two are rarely the same.
You should see: Two named terms, for example: input tokens on the retrieval-augmented answering stage dominate cost; serial tool calls dominate latency.
- Match one lever to each dominant term — caching, context curation, output contracts, effort tuning, streaming, parallel tool calls, right-sizing with cascade, or batch — and reject at least one tempting lever that does not move your dominant term, with a sentence on why
Why: Lever selection is the examined skill, and the rejected lever proves the selection was reasoned rather than fashionable.
You should see: One lever per term with a mechanism sentence, plus one explicit rejection, for example: caching rejected — inputs are unique per request, no stable prefix.
- State what each chosen optimisation spends to save what (the frontier framing)
Why: Optimisations move along the cost-performance frontier; naming the spend is what keeps the trade-off honest.
You should see: One sentence per lever: spends X (turnaround, capability headroom, engineering effort) to save Y (unit cost, tail latency).
- Define the re-gate: which 4.1 metrics must hold, on which dataset, before the optimisation ships
Why: The re-gate is the difference between an optimisation and an unmeasured regression.
You should see: A named metric set, the eval dataset version, and the rule: ships only if every guard metric holds.
- Project the saving at volume (per task multiplied by projected volume) and compare it against the measured baseline
Why: Per-task savings that vanish at real volume, or engineering costs that exceed the annual saving, are findings the projection surfaces.
You should see: A per-task delta, a volume projection, and an honest comparison against the baseline including the cost of doing the work.
Sources
- Prompt caching — Anthropic
- Batch processing — Anthropic
- Reducing latency — Anthropic
- Effort — Anthropic
- Choosing the right model — Anthropic
- Claude Certified Architect Professional Exam Guide (v1.0) — Domain 4 — Anthropic