Optimising Context Windows and Token Usage

What You Need to Know

The context window is everything Claude sees in one request, counted in tokens. Every token in it is billed on every request it appears in. The window holds four things:

• the system prompt
• every message in the history, including tool results, images and documents
• the tool definitions
• the output being generated, thinking tokens included

Take a pharmacy whose counter staff use a Claude assistant. Its system prompt, stock-lookup tool, conversation and leaflet pages are all in the window, billed on every request.

Counting the Window

Two documented, free instruments count tokens. The usage field on every response reports what the request consumed. With caching, input splits into plain, cache-read and cache-write tokens, and all three count toward the window. The token counting API estimates a request before you send it. Its count is inexact, and it has its own rate limits. Pre-flight estimation belongs in the design: the pharmacy counts its largest leaflet request before launch.

Overflow is a hard failure with two distinct cases. Oversized input is rejected outright. On current models, generation that reaches the limit stops early with its own documented stop reason. Both are detectable, so alert on them. Budget so routine requests never come near either.

Context Rot: Capacity Is Not a Strategy

Window sizes are tiered by model, and output has its own ceiling. A bigger window is capacity, not a strategy. Anthropic's docs explain why: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot. This makes curating what's in context just as important as how much space is available." Anthropic's engineering guidance calls context an attention budget. Its goal is the smallest possible set of high-signal tokens that maximises the likelihood of the desired outcome.

So ask whether every token earns its place, not whether it fits. Loading the pharmacy's whole leaflet library into a million-token window fails three ways:

• rot degrades answer quality
• every request pays for the full load
• latency grows with input size

The Replay Bill

Each turn resends the whole history. Over 20 turns at the pharmacy counter, a 50k-token history is 50k input tokens twenty times. At volume, careless context becomes the dominant cost line, ahead of model choice. That makes context strategy a Domain 2 economics decision. Curation, caching and compaction each attack a different part of the replay bill.

The Management Toolkit

• Curate first. Send the fields, not the file: the stock fields a question needs, not the whole stock list. Summarise verbose history and strip formatting noise. Put longform documents, such as a leaflet, at the top of the prompt, above the query, per Anthropic's long-context guidance. The cheapest optimisation is the token that never enters the request.
• Retrieve instead of stuffing. When the source corpus exceeds sensible context, retrieval brings only the relevant passages for each request, such as the two leaflets a question needs. Choosing retrieval is a Domain 2 cost-and-context judgement; designing the RAG pipeline is Domain 3.
• Compact long conversations. Server-side compaction (beta, Claude 4.6-generation and later models) summarises earlier turns into a compaction block when input tokens reach a trigger threshold. The conversation can then continue past the window limit. It is Anthropic's recommended primary strategy for long-running conversations and agentic sessions, such as a full shift at the counter.
• Edit agentic context. Context editing (beta) clears the oldest tool results (and optionally thinking blocks) server-side once a threshold is reached. It replaces them with placeholders. Use it where the full history adds nothing, such as stock lookups from hours earlier.
• Trim the margins with structure. Tight output contracts shrink responses. Lean, deferred tool definitions shrink fixed overhead.

Truncation passes for optimisation. It cuts context to a fixed token count, whatever the task needs, and loses quality silently. Curation selects by signal; truncation cuts by position. Cut a long leaflet to its opening and the later sections vanish unflagged. Optimisation must preserve task-required information.

In the exam

The exam tests whether you treat the window as a budget under active management or as capacity to be filled. When cost and quality problems share one stem, tell them apart. "Enable caching" offered as the fix for window overflow or late-conversation quality decline is a planted wrong answer. So is truncation: the guide's own sample distractor cut a policy to its first 1,000 tokens.

Current window sizes, output ceilings and stop-reason identifiers are in the Domain 2 quick reference.

Exam Traps

Practice Scenario

Build Exercise

Sources

• Context windows — Anthropic
• Token counting — Anthropic
• Compaction — Anthropic
• Context editing — Anthropic
• Effective context engineering for AI agents — Anthropic
• Claude Certified Architect Professional Exam Guide (v1.0) — Domain 2 — Anthropic