Model Selection (2.1)
Selection is constraint satisfaction, not prestige. Five axes: capability (a measured bar), latency (compounds across chains and loops), cost (price × tokens × volume), context/output limits, lifecycle and platform availability. Constraints eliminate candidates before preference gets a vote.
- Two documented starting points: efficiency-first (start Haiku-class, upgrade on a measured gap) or capability-first (start Opus-class, optimise down). Either way, benchmark evals on representative data are the gate.
- Right-size per pipeline stage, not per application. The smallest model that clears each stage's measured bar.
- Effort parameter (current models) trades intelligence for latency and cost within one model, often a better lever than switching tiers.
- Escalation cascade (our shorthand; Anthropic documents the pattern as routing): default to the small model, escalate on low confidence or failed validation. Near-big-model quality at near-small-model average cost — the keyed answer in cost-vs-quality stems.
- Lifecycle: every model ID is a pinned snapshot; models move active → legacy → deprecated → retired, with at least 60 days' notice. Pin versions, monitor notices, re-run evals before migrating. A model change is an architecture change.
- Snapshot checked 14 October 2026 (re-verify before relying on it): Fable 5.1 $10/$50, Opus 5.5 $4/$20, Sonnet 5.5 $2/$10, Haiku 5.5 $0.10/$0.50 per MTok in/out (Haiku 5.5 rises to $0.50/$2.50 for prompts over 100,000 tokens); 1M-token windows and 128k max output on all four. The previous generation (Fable 5, Opus 5, Sonnet 5) and Haiku 4.5 remain available as legacy models.
System Prompts, Templates & Guardrails (2.2)
The system prompt is a contract: role, audience, standing policy, output contract. Per-request data belongs in user turns. The split is doubly load-bearing — stable behavioural authority plus a cacheable prefix.
- Altitude rule: between hardcoded brittle logic and vague guidance — specific enough to steer behaviour, flexible enough to leave the model strong heuristics.
- Golden rule: if a colleague with minimal context would be confused, so will Claude. Give the motivation behind unusual rules.
- Escape hatch: an explicit null path plus permission to admit uncertainty. An output contract without one forces invention — the first-line hallucination control.
- Templates: XML-tagged sections (instructions, context, examples, per-request input) with named variable slots — diffable, reviewable, evaluated before deploy, rolled back on regression. One reviewed template beats ten drifting hand-copies.
- Guardrails in three layers: instruction-level (scope, refusal rules, escape hatch) → input-side (validate, sanitise, delimit untrusted content as data) → output-side (schema plus business-rule checks before anything downstream consumes the answer). Instructions shape behaviour probabilistically; only code guarantees it.
- Every refusal needs a route. A block with no fallback turns a safety control into an outage.
Zero-Shot, Few-Shot, Chain-of-Thought (2.3)
The ladder runs in cost order: clear zero-shot instructions → few-shot examples (3–5) → chain-of-thought. The exam rewards the cheapest technique that fixes the observed failure, so diagnose first.
| Failure symptom | Cheapest effective fix |
|---|---|
| Format wrong or drifting | Few-shot with an exact-format example |
| Boundary misclassification | Few-shot with near-miss/boundary examples |
| Multi-step reasoning errors | CoT or built-in thinking |
| Stated rules ignored | Instruction clarity and structure first |
| Any of the above | Never a model upgrade first |
- Few-shot sets must be relevant, diverse, structured (wrapped in example tags). Happy-path-only sets teach the wrong pattern.
- CoT: separate reasoning from the answer with tags, or downstream parsing breaks. Wrong for latency-critical trivial tasks.
- Built-in thinking (mid-2026): adaptive on 4.6-generation and later models; thinking tokens bill as output and count toward max_tokens. Prompted CoT stays right where reasoning must be auditable in the response body.
- Current state (checked 6 October 2026): the prompting docs now say to rely on thinking on Claude Fable 5.1, Fable 5, Opus 5.5, Opus 5 and Sonnet 5.5, where a prompt asking for written-out reasoning (for example in
<thinking>tags) may be declined. On the exam, prompted CoT is still the answer. - Reasoning inside one call is 2.3. Decomposing into chained calls is Domain 1 architecture.
Context Windows & Token Usage (2.4)
Everything occupies the window: system prompt, history, tool definitions, tool results, images, and the output being generated — thinking tokens included. Every token is billed on every request it appears in: the replay bill.
- Instruments: the usage field on every response (input_tokens, cache_read_input_tokens, and cache_creation_input_tokens all count toward the window); the free token counting API for pre-flight estimates.
- Overflow: input too large → 400 "prompt is too long"; generation hitting the limit → stop_reason model_context_window_exceeded.
- Context rot: accuracy and recall degrade as token count grows. Target the smallest set of high-signal tokens — capacity is not a strategy.
- Toolkit, in order: curate (send the fields, not the file; longform documents above the query) → retrieve instead of stuffing → server-side compaction (the recommended primary strategy for long sessions) → context editing (clears stale tool results) → tight output contracts and lean tool definitions.
- Truncation ≠ curation. Cutting to a fixed count discards by position; curation selects by signal.
- Caching never buys window space — cached prefixes still occupy the window. "Enable caching" as the fix for overflow or rot is a planted wrong answer.
Prompt Reuse: Caching, Modular Prompts, Skills (2.5)
Three mechanisms, three problems: caching (the same prefix resent at full price), modular prompts (drifting hand-copies of shared text), Skills (occasional deep expertise without a standing cost).
- Cache prefix order: tools → system → messages. A change at any level invalidates that level and everything after it — front-load stability. A timestamp early in the prompt defeats the whole cache.
- Mechanics (checked 14 October 2026): up to 4 explicit breakpoints (or automatic caching via a top-level cache_control field); 5-minute TTL default, 1-hour option, refreshed free on use; minimum cacheable length 512 tokens (Fable 5.1, Opus 5.5, Sonnet 5.5, Haiku 5.5, Opus 5, Fable 5), 1,024 (Sonnet 5), 4,096 (Haiku 4.5). Prompts below the minimum run uncached without an error.
- Economics: writes cost 1.25× (5-min) or 2× (1-hour); reads 0.1× — roughly 90% off. Break-even after one read (5-min) or two (1-hour). At volume, caching cuts both cost and time-to-first-token; on sporadic traffic outside the TTL, the write premium can exceed the savings.
- Caching is not persistence, window relief, or a quality lever.
- Skills: a folder with SKILL.md — name (max 64 chars) plus a trigger-bearing description (max 1,024). Progressive disclosure: ~100-token metadata always loaded; the body (under 5k tokens) loads on trigger; bundled resources cost nothing until read. Custom Skills do not sync across surfaces (API, claude.ai, Claude Code).
- Placement rule: standing policy → cached system prompt; on-demand expertise → Skills; per-request data → messages.