Quick Reference: Domain 5 — Model Selection and Optimisation

Claude sits on an autoregressive model, pretrained "to predict the next word, given the previous context of text in the document". At inference it emits one token at a time, each conditioned on everything before it, its own output included. Three consequences carry the marks:

• Generation is serial, so response time scales with output length. Input length drives the prefill and therefore time to first token; output length drives the long tail. Reach for the wrong one and you answer a latency complaint with a capability lever.
• There is no lookahead and no revision. A token, once emitted, is part of the record. That is why a stated output contract and a worked example placed before the task beat an instruction to fix the format afterwards.
• Each token conditions the next, so an early wrong turn compounds instead of self-correcting. The model is continuing what it has written, not sampling afresh at each position.

Thinking is the same mechanism run before the visible answer: reasoning tokens are generated first, billed as output, and counted against max_tokens.

A token is a unit of the model vocabulary, not of human text: tokens "can correspond to words, subwords, characters, or even bytes (in the case of Unicode)". Code, JSON, tables and non-English text tokenise differently from prose, so a ratio calibrated on paragraphs misprices a workload of log lines. Don't rely on a tokens-per-word ratio. The documentation carries two such ratios that disagree with each other, roughly 3.5 characters per token in the Glossary against roughly 4 characters or 0.75 words in the Pricing FAQ, and that disagreement is the lesson.

Three instruments, three jobs, and swapping one for another is a wrong answer by construction:

The counting endpoint accepts the same structured inputs as a real message (system prompt, tools, images, PDFs) and returns a single input_tokens field. Its answer is explicitly "an estimate" and it does not apply caching logic.

Dated current-state note (August 2026): "Claude 4.7 and later models and Claude Mythos Preview use a newer tokenizer. The same input text produces approximately 30 percent more tokens than on earlier models." The endpoint "returns the count under the tokenizer of the model you pass", so a count taken against last year's model is the wrong number for this year's, for both the bill and the window headroom. Recount whenever you change models.

The window is not an input allowance. Everything competes for the same space: "the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions. The output Claude generates for the turn, including its extended thinking, counts too."

Two corollaries are routinely got backwards. Caching creates no headroom: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count." And bigger is not automatically better: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot."

Overflow has two behaviours, and the discriminator is whether the input alone is too big:

Dated current state (August 2026): the newer behaviour can be turned on for earlier models — the docs say to "use the model-context-window-exceeded-2025-08-26 beta header" to opt in. The stop reason is also "currently typed only in the SDKs' beta namespace", so typed access goes through client.beta.messages.

Both branches are arithmetic against a budget and both have budget fixes: send less, retrieve fewer and better passages, split the job, or move to a larger window. Neither is a prompt fault and neither is fixed by a more capable tier. Rolling first-in-first-out truncation of old messages is chat-interface behaviour, not API behaviour, and it is the standing distractor here.

The single most durable claim in the domain, because it is a property of the service rather than of a release: "Even with temperature set to 0, the results will not be fully deterministic and identical inputs may produce different outputs across API calls. This applies both to Anthropic's first-party inference service and to inference through third-party cloud providers."

Temperature zero buys consistency, not reproducibility. Never build a component whose correctness requires two identical calls to return identical bytes. Constrain the shape instead: a worked example, an explicit output contract, validation at the boundary. Store what has to stay stable rather than regenerating it and comparing.

Dated current-state note (August 2026): the sampling dials are not merely weak, they are rejected. The API parameter deprecations table lists temperature, top_p and top_k as "Deprecated (Claude Opus 4.7 and later)", returning "a 400 error when set to a non-default value on Claude 4.7 and later models and Claude Mythos Preview", with the recommended replacement "Omit and use prompting to guide model behavior." On older models the restriction applies only while thinking is on. The documentation set is not internally consistent here: the Reducing latency page still suggests experimenting with temperature and has not been updated. Treat the deprecations table and the Thinking page as the authority.

The guide names these four together, and the durable knowledge is which complaint each one answers. They are not interchangeable.

Extended thinking is the older interface: the legacy Claude Haiku 4.5 still uses it, and the newest models reject it.

Effort is the workhorse and the first lever to try. The default depends on the model. As of 14 October 2026, "Most Claude models default to high effort", while "Claude Opus 5.5 and Claude Haiku 5.5 default to medium". "Setting effort to the model's default ("medium" on Claude Opus 5.5 and Claude Haiku 5.5, "high" on other models) produces exactly the same behavior as omitting the effort parameter entirely." Two properties earn it first place. It "applies to every output token", so "it works whether or not thinking is enabled". And it covers tool calls as well as text, so "Lower effort also means fewer and terser tool calls" rather than merely shorter prose. It is "a behavioral signal, not a strict token budget".

Dated current-state note (September 2026): adaptive thinking is already on with no configuration needed on Claude Opus 5.5, Opus 5, Sonnet 5.5, Sonnet 5, Fable 5.1, Fable 5 and the Mythos models, and as of 14 October 2026 on Claude Haiku 5.5 too. On those models the thinking display field defaults to "omitted": thinking blocks come back with an empty thinking field until you set display: "summarized", and no setting returns the raw chain of thought. Read the thinking blocks rather than asking for the reasoning in the answer, because on Claude Fable 5.1, Opus 5.5, Opus 5, Sonnet 5.5 and Fable 5 that request can be refused with stop_details.category: "reasoning_extraction", a category billed even when the refusal arrives before any output. Fast mode is a research preview delivering "up to 2.5x higher output tokens per second compared to standard speed", limited to Claude Opus 5.5, Claude Opus 5 and Claude Opus 4.8, on the Claude API only, at premium pricing: $8 in and $40 out per MTok on Claude Opus 5.5, $10 and $50 on the other two. Its gain is in OTPS, "not time to first token (TTFT)", so it tells on long streamed answers and barely registers on short ones. It is unavailable with the Batch API or a Priority Tier commitment, and switching between fast and standard speed invalidates the prompt cache.

The selection rule, in three sentences. A hard problem answered shallowly wants more reasoning. A routine problem that costs too much wants less effort. An answer that is already correct but arrives too slowly for a person watching wants faster delivery, meaning streaming first and fast mode where it exists, not a different model.

The guide's vocabulary is zero-shot (an instruction, no demonstration), single-shot (one worked input-output pair) and multi-shot (several). The current documentation calls the same family "few-shot or multishot prompting" and makes the durable claim: "Examples are one of the most reliable ways to steer Claude's output format, tone, and structure."

The quality tests are named. Examples should be Relevant ("Mirror your actual use case closely"), Diverse ("Cover edge cases and vary enough that Claude doesn't pick up unintended patterns") and Structured (wrapped in <example> tags so Claude can tell them from instructions), with "Include 3-5 examples for best results."

Diversity is the one candidates underrate: a set of examples sharing an incidental property teaches that property. If every example happens to be a clean run, the model learns that notes never report an exception.

When to reach for examples: a failure of form or of boundary, meaning inconsistent structure, unclear classification edges or drifting tone. Not a failure of budget (a request that will not fit) or of fact (something the model cannot know). Examples cost input tokens on every call, which is exactly why a stable example block at the front of a prompt is a caching candidate.

Underneath every Claude application is one HTTP POST and one JSON document. Three headers on POST /v1/messages (plus anthropic-workspace-id with a multi-workspace key): the credential as Authorization: Bearer <key> (as of September 2026 the older x-api-key header is the legacy fallback, still accepted), anthropic-version (2023-06-01) and content-type. The response is ordinary JSON with an id, a role, an array of typed content blocks, a stop_reason and a usage object. Failures arrive in a fixed envelope: a top-level type of error, a nested error object always carrying a type and a message, and a request id that also rides on the request-id response header.

Anthropic ships client SDKs for seven languages (Python, TypeScript, C#, Go, Java, PHP, Ruby) plus the ant CLI: "Each SDK provides idiomatic interfaces, type safety, and built-in support for streaming, retries, and error handling." Unpacked, that is credential and version headers, typed requests and responses, a typed exception hierarchy, automatic retry of the transient class, a default timeout, parsed SSE events with an accumulation helper, one interface over sync and async, _request_id on responses, auto-pagination, and dedicated client classes for the cloud platforms. None of it is interesting. All of it is yours the moment you drop the dependency.

What it emphatically does not change is where the items live. No SDK-only models, no SDK discount, no higher or waived rate limits, no uptime guarantee, no immunity from an error class. Models, prices, rate limits and contractual terms are properties of the API and of your agreement, not of a package. The exam-safe formulation: the SDK changes the ergonomics of calling the API and nothing about the API itself.

Branch on the class, most specific first, never on message text, which is not a contract:

Two defaults are misread more often than anything else here. The retried set: "Certain errors are automatically retried 2 times by default, with a short exponential backoff. Connection errors (for example, because of a network connectivity problem), 408 Request Timeout, 409 Conflict, 429 Rate Limit, and >=500 Internal errors are all retried by default." The timeout: "By default requests time out after 10 minutes." They compose, because "requests that time out are retried twice by default", so worst-case wall clock is roughly the timeout multiplied by the attempt count.

Not every 429 is transient. As of September 2026 a usage tier's monthly spend cap also returns one, with no retry-after header and an error.details.error_code of enforced_spend_limit_reached. It "keeps failing until access resumes", the SDK's own retries included. Read that field before queueing a RateLimitError for retry.

Stacking your own loop on top does not add resilience. It multiplies attempts, re-sends deterministic failures that will never succeed, and pushes a rate-limited endpoint further into the limit. Set max_retries and the timeout deliberately at one layer, and let the other decide policy: degrade, queue, or fail loudly.

Construct one client and reuse it. The client owns an HTTP connection pool. A client built inside every request handler throws that pool away per call, paying a fresh TLS handshake and multiplying open sockets under concurrency. It presents as latency that grows with traffic and connection errors that look like an upstream problem, so a stem describing exactly those symptoms is usually describing this defect.

The reliable item shape is a developer proposing the right transport at the wrong layer.

MDN on SSE: "This is a one-way connection, so you can't send events from a client to a server." MDN on WebSockets: they make it "possible to open a two-way interactive communication session between the user's browser and a server". Anthropic draws the layer line in its own hosting guidance: an application container "exposes an HTTP or WebSocket endpoint" to its clients, while separately "The SDK needs outbound HTTPS to api.anthropic.com."

No documented Anthropic API surface accepts a websocket connection from a client, so "open a websocket to Anthropic so the user sees text appear" is wrong at the protocol level rather than merely suboptimal. The reverse is equally testable: streaming from the API needs no websocket in your front end, and a websocket in your front end gives you no streaming from the API. Independent decisions at different layers.

A stream can fail after it has already succeeded. A 200 opens the stream, and "The API may occasionally send errors in the event stream. For example, during periods of high usage, you may receive an overloaded_error", delivered as an event: error frame mid-stream. Error handling wrapped around the call but not the iteration produces a silently truncated answer with no exception. Two more mechanics. Streaming is sometimes obligatory: the trigger is expected duration rather than any published max_tokens figure, and the Python SDK "will throw a ValueError if a non-streaming request is expected to take longer than approximately 10 minutes" (passing stream=True or overriding timeout disables that check). And "The token counts shown in the usage field of the message_delta event are cumulative", so they are read rather than summed.

The ordering is the durable part and is stated first-party. As of 14 October 2026 the cost guidance reads: "From lowest to highest cost and capability, the current models are Claude Haiku 5.5, Claude Sonnet 5.5, Claude Opus 5.5, and Claude Fable 5.1 (the frontier model)". A month earlier the same sentence began with Claude Haiku 4.5 and Claude Sonnet 5. That drift is the reason to learn the family order and not the version numbers. Comparative latency runs the other way, so the three axes the guide names, quality, latency and cost, are not independent.

Everything in this table is dated and will move. It follows the Models overview comparison table:

From the 4.6 generation the dateless format is itself a pinned snapshot, so every id in the table is already pinned. Older models carry a snapshot date in the id instead: the legacy Claude Haiku 4.5 is claude-haiku-4-5-20251001, and claude-haiku-4-5 is a convenience pointer that resolves to it.

Documented characters, which follow from the ordering: Haiku is for "The lowest latency and price": real-time work, high-volume processing and sub-agent tasks. Sonnet is "The best combination of speed and intelligence" for code generation, data analysis and agentic tool use. Opus is for "Complex agentic coding and enterprise work". Fable 5.1 is "For demanding reasoning and long-horizon agentic work".

Capability was not uniform when the guide was written, which is why it names adaptive thinking support in its own scope. Claude Haiku 4.5, the fastest model then and a legacy model now, rejects adaptive thinking, carrying the older extended thinking form instead, and it has no effort parameter either. As of 14 October 2026 all four current models have adaptive thinking, Claude Haiku 5.5 included. A design that depends on the model regulating its own reasoning depth still rules out any model without it, Haiku 4.5 among them, on capability before cost is discussed.

Start from the criteria the documentation names: Capabilities, Speed, Cost and Effort. The fourth is the one candidates skip, and it is a lever inside a model rather than a choice between models: "Several Claude models support an effort parameter that trades intelligence for latency and cost within a single model. Tuning effort is often a better lever than switching models."

Two documented starting strategies bracket the space, and neither is a default:

• Efficiency-first: start with the fastest tier, test thoroughly, upgrade "only if necessary for specific capability gaps". Home ground: prototyping, tight latency bars, cost-sensitive and high-volume straightforward work.
• Capability-first: start with "the strongest starting point for your task", optimise the prompt for it, then consider "increasing efficiency by lowering effort or downgrading models over time". That is not the same as the most capable model: as of September 2026 the page starts on Claude Opus 5.5 and moves to the frontier Claude Fable 5.1 only if evals at xhigh or max effort still fall short. Home ground: complex reasoning, "tasks requiring nuanced understanding", accuracy outweighing cost.

Then compare in the right currency and on the right sample. Currency: "You pay for completed tasks, though, so compare models on cost per completed task. A more capable model finishes a task with less work: fewer turns, less searching, less re-reading of its own context, and less backtracking." Sample: "Price the tail of your workload, not the median: compare models on the hardest tenth of your tasks, not the typical one." And the line that kills every price-list argument: "The ranking flips by workload, and no price list tells you which way."

The eval is the gate. "Create benchmark tests specific to your use case - having a good evaluation set is the most important step in the process", run "with your actual prompts and data". A benchmark table or a price cut is not a migration gate, because published numbers are measured on somebody else's tasks. Vary one thing at a time, run each effort level in a separate session (changing top-level effort mid-session starts the cache over and distorts the cost comparison), and remember a prompt tuned for the old model is not a neutral test of the new one.

A sweep can end in a policy, not one level. As of September 2026 the cost guidance documents one for checkable work: "run every task at a low setting and re-run only the failures at a higher one". On its coding benchmark that slightly beat running everything at high, for a little over half the cost. It needs a failure signal you trust, and every first-pass failure takes two runs of wall-clock time.

The vocabulary is exact and testable. Legacy is not Deprecated and Deprecated is not Retired:

Anthropic commits to "providing at least 60 days' notice before model retirement for publicly released models", and "Every Claude model ID is a pinned snapshot". Two planning facts follow: a retirement date is a deadline on your roadmap, so staying on what works has an expiry; and because ids are pinned, nothing changes underneath you unless you change it, which is what makes a planned migration possible at all.

A new model is a new system, and the differences come in three kinds:

• Behaviour shifts in named ways: default response length, how literally instructions are read, tool-calling tendency, readiness to delegate to subagents.
• Interface changes break code outright: manual extended thinking with budget_tokens, assistant prefill, and non-default temperature, top_p or top_k all return a 400 on the newest models.
• Cost changes arrive even when nothing breaks, because a prompt written to compensate for an older model is followed literally by a newer one.

Interface changes fail loudly; the expensive ones do not. Swapping the model string in one place and carrying the prompts, effort settings and thinking configuration across unchanged is the shape of the quiet failures. Run the eval suite, audit the prompts against the model you are moving to, re-sweep effort, then shadow on a traffic slice before cutover.

A cost model is arithmetic over usage multiplied by real traffic: "price the five priced token counts in each response's usage at their own rates, summed across the task's requests" (wording as of September 2026; cache writes bill at the 5-minute or 1-hour multiplier). The top-level four are input_tokens, cache_creation_input_tokens, cache_read_input_tokens and output_tokens.

Two structural facts drive almost every decision after that:

• Output is priced several times input on every model. A workload that generates a lot relative to what it sends is output-dominated; one that sends a lot relative to what it generates is input-dominated; the fix for each is different.
• input_tokens does not mean all of the input. It counts the tokens after the last cache breakpoint, so on a cached request the real input volume is the sum of all three input fields. Read it alone on a heavily cached workload and you will conclude the prompt shrank when it did not.

Name the dominant axis before choosing a lever. A long standing prefix re-sent hundreds of times a day with short answers is an input problem, and shortening the answers optimises the wrong axis entirely. A workload sending two hundred tokens and generating four thousand is an output problem, and caching the prompt moves almost nothing. Thinking tokens bill as output and are reported under output_tokens_details.thinking_tokens, so a reasoning-heavy workload is output-dominated even when its visible answers are short. Do the arithmetic on one representative day of real traffic, never on one request.

Tracking is engineering, not accounting. Three layers, three jobs. Per request, log the counts beside the request id and a feature label, on successes as well as failures. Per feature, alert on the ratio you expect: a cache hit rate, an input-to-output ratio, a cost per completed task. A regression should announce itself rather than turn up on an invoice. Per organisation, the Usage and Cost Admin API is the authoritative aggregate, grouping "by API key, workspace, model, service tier, context window, data residency, or speed"; it needs an Admin API key, is "unavailable for individual accounts", surfaces data within about five minutes, and supports polling once a minute.

Measuring a change against a monthly invoice is the distractor. An invoice aggregates every workload and every traffic fluctuation, so nothing can be attributed to anything.

Domain 2 owns how caching is configured. Domain 5 owns whether it pays, and the economics are asymmetric. Key on the shape; look up the numbers. A write costs more than not caching, a read costs a fraction, and the longer lifetime costs more to write than the shorter one.

"A cache hit costs 10% of the standard input price, which means caching pays off after one cache read for the 5-minute duration (1.25x write), or after two cache reads for the 1-hour duration (2x write)."

The rule in words: caching pays when a large, stable prefix is re-sent often enough within its lifetime to amortise the write. Each of the four conditions has teeth.

• Large: prompts below the model minimum "are processed without caching" and return no error, so a small prefix silently does nothing. The minimums as of 14 October 2026: 512 tokens on Claude Fable 5.1, Mythos 5.1, Opus 5.5, Opus 5, Sonnet 5.5, Fable 5, Mythos 5 and Haiku 5.5; 1,024 on Claude Opus 4.8, Sonnet 5, Sonnet 4.6 and Sonnet 4.5; 2,048 on Mythos Preview, Opus 4.7 and Haiku 3.5; 4,096 on Claude Opus 4.6, Opus 4.5 and Haiku 4.5.
• Stable: the prefix must match byte for byte, so a timestamp, a per-request identifier, an unsorted serialised object or a varying tool list ahead of the breakpoint drives the hit rate to zero silently.
• Often enough: a prefix used once loses money outright, paying the write premium and never collecting the read discount.
• Within its lifetime: the clock is "measured from the start of the request that writes or reads the cache entry, not from response end", it "is refreshed at no additional cost each time cached content is used", and generation time counts against it. A response that streams for minutes has eaten most of the shorter window before the next request begins.

Check-pointing a growing conversation. The request renders in a fixed order, tools then system then messages, and a change at one level invalidates that level and everything after it. The settled transcript is the stable part and the new turn is the volatile part, so the breakpoint belongs at the boundary and must advance as it moves; automatic caching does that advancing for you. With explicit breakpoints, two limits shape the design: at most 4 breakpoints per request, and a 20-block lookback window per breakpoint. A conversation that grows further past the last write than that lookback will miss, and the fix is a second breakpoint placed closer and from the start. Anything inserted mid-prefix invalidates everything downstream, which is why appending is cheap and editing history is expensive. Configuration counts as prefix: the thinking configuration and the resolved effort level render into the prompt, and switching between fast and standard speed does the same. As of September 2026 a per-message effort change (beta, supporting models) is the exception: an effort-only system message sets the new level from the next user turn and the cached prefix still matches.

Verifying is part of the change. A cache_read_input_tokens of zero where you expected a hit is a silent invalidator, not a pricing surprise.

Anthropic splits them cleanly. Free wins "cut spend without touching quality: prompt caching, token hygiene, a prompt audit against the model you are running, batch processing at 50% off for work that can wait up to 24 hours, and workspace spend limits as the backstop." Trade-offs "exchange cost for intelligence: model choice, effort, output caps and task budgets, an elapsed-time clock, and multi-model architectures."

Take the free wins first, in this order:

1. Prompt caching is the largest single measured lever: as of September 2026, caching cut agent-loop cost by a factor of 2.7 to 5.3 on Anthropic's benchmarks and cut a small triage agent's bill by 83 per cent, and "cache reads are routinely the largest single component of task cost".
2. Token hygiene, meaning pruning stale tool output, retrieving fewer and better passages, dropping dead transcript and right-sizing images. In the measured triage run it took that 83 per cent saving to 88 per cent.
3. A prompt audit against the model you actually run, removing instructions written for an older one that a newer one now follows literally.
4. Batch processing at a 50 per cent discount on both input and output, in exchange for asynchronous delivery inside 24 hours. Route anything nobody is waiting on through it, and nothing that anybody is.

Then the trade-offs, where effort is the direct dial and a cheaper tier is legitimate for tasks that pass an eval with a stated pass bar.

And then the one that is not a lever at all. As of September 2026: "A task budget saves money, because the model sees it. max_tokens is a safety cap: lowering it cut cost per attempt without lowering cost per solved task." It "caps a single response, invisibly to the model, so lowering it does not make the model economize. The turns that needed the room are discarded and still billed." In Anthropic's internal measurement a tight cap cut off a large share of attempts, and very few of those still passed. Attempts got cheaper and fewer tasks got solved, so cost per solved task barely moved. The correct handling inverts the instinct: set max_tokens high enough that nothing is cut off (the page suggests 64,000 for agentic work, or the 128,000 maximum when one cut-off attempt is costly), "treat stop_reason: max_tokens as a failure", and take the saving from levers the model can respond to.

A task budget is the lever the model can see. As of September 2026 it is a beta, set as task_budget inside output_config, that shows the model a running token countdown for the whole agentic loop, so the model trims low-value work and wraps up rather than being cut off mid-action. It is advisory, leaving max_tokens as the hard per-request ceiling, and a mid-task change invalidates the cached prefix, so set it once on the first request. Size it from your own measured per-task usage. The Task budgets page suggests starting at the 99th percentile of that usage, then testing up or down.

Domain boundary: Messages-API mechanics, streaming event shapes and how cache_control is configured belong to 2.3; pinning and recording a snapshot is 2.6; prompt craft is Domain 6; failure diagnosis is Domain 4. Domain 5 owns the economics and the choice.