LLM Fundamentals

What You Need to Know

Claude sits on an autoregressive language model: it writes text one token at a time. A token is a unit of the model's vocabulary. The context window is all the text the model can reference while it writes, including what it writes. Sampling is how the model picks each next token; temperature sets how widely. Model options change how far it reasons, how hard it works and how fast tokens arrive.

Take a supermarket that answers shoppers in an online chat. Each request carries the store's policy pack, a catalogue extract, the chat so far and the new question. Shoppers ask about opening hours, stock and refunds.

Next-Token Generation, and the Three Things That Follow

Anthropic's glossary says models like this are pretrained "to predict the next word, given the previous context of text in the document". At inference the model emits one token at a time. Each token is conditioned on the whole sequence so far, including the tokens it has just produced. Three consequences follow.

Generation is serial, so response time scales with how many tokens come out. Input length affects the prefill and therefore time to first token. Output length affects the long tail. A long policy pack delays the first word a shopper sees, and a long reply delays the last.

There is no lookahead and no revision. Once a token is emitted it is part of the record. The model cannot go back and restructure an answer it started badly. So an output contract stated up front beats a correction added afterwards. A worked example placed before the task beats an instruction to fix the format if it drifts. If the supermarket wants each refund reply to open with the amount, it shows that shape first.

Each token conditions the next, so an early wrong turn compounds rather than self-corrects. The model is not sampling independently at each position. It is continuing what it has already written.

Thinking is the same mechanism applied before the visible answer. Reasoning tokens are generated first. They are billed as output tokens and count toward max_tokens alongside the response text.

Tokens Are Measured, Never Estimated

Tokens are not units of human text. They "can correspond to words, subwords, characters, or even bytes (in the case of Unicode)". Code, JSON, tables, identifiers and non-English text all tokenise differently from English prose, so a rule of thumb calibrated on paragraphs misprices a workload of log lines. The supermarket's catalogue extract, full of product codes, will not price like its policy prose.

The documentation set carries two different rules of thumb. The glossary says roughly 3.5 characters per token. The pricing FAQ says roughly 4 characters or 0.75 words. The disagreement is the lesson, not a fact to memorise.

Two instruments answer two different questions. Before the request, POST /v1/messages/count_tokens accepts the same structured inputs as a real message, including system prompt, tools, images and PDFs. It returns a single field:

import anthropic

client = anthropic.Anthropic()

MALT_HOUSE_STYLE_GUIDE = "You are the malt house's shift-log assistant."
steeping_log = "Batch 214: steep started 06:40, water at 14 degrees."

count = client.messages.count_tokens(
    model="claude-sonnet-5-5",
    system=MALT_HOUSE_STYLE_GUIDE,
    messages=[{"role": "user", "content": steeping_log}],
)
print(count.input_tokens)  # {"input_tokens": 14} on the wire

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const MALT_HOUSE_STYLE_GUIDE = "You are the malt house's shift-log assistant.";
const steepingLog = "Batch 214: steep started 06:40, water at 14 degrees.";

const count = await client.messages.countTokens({
  model: "claude-sonnet-5-5",
  system: MALT_HOUSE_STYLE_GUIDE,
  messages: [{ role: "user", content: steepingLog }]
});
console.log(count.input_tokens);

Counting is free. Token counting and message creation "have separate and independent rate limits", so counting costs neither money nor Messages-API headroom. Its answer is explicitly "an estimate", and it does not use caching logic. After the request, the usage object on the response is the ground truth you bill against.

The third point catches more migrations than the other two. Token counts are tied to a tokenizer, and tokenizers change between model generations. Dated current state (August 2026): "Claude 4.7 and later models and Claude Mythos Preview use a newer tokenizer. The same input text produces approximately 30 percent more tokens than on earlier models". The count endpoint "returns the count under the tokenizer of the model you pass". So a count measured against last year's model is wrong for this year's, for both the cost model and the context-window headroom. Count against the model you will call, and recount whenever you change models.

The Context Window Is One Budget, Shared

The window is not an input allowance. It "refers to all the text a language model can reference when generating a response, including the response itself". Everything competes for the same space: "the system prompt, every message in messages (including tool results, images, and documents), and your tool definitions". And: "The output Claude generates for the turn, including its extended thinking, counts too". For the supermarket, the policy pack, catalogue extract, history and reply share one budget.

Two corollaries are routinely got backwards. First, caching creates no headroom: "Cached prompt prefixes still occupy the context window: prompt caching changes what you pay for those tokens, not whether they count". Second, thinking is not free space: its tokens are drawn from max_tokens and billed as output. Bigger is also not automatically better: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot".

Overflow has two behaviours, split by whether the input alone is too big.

Both branches are arithmetic against a budget, with budget fixes: send less, retrieve fewer and better passages, split the job, or move to a model with a larger window. Neither is a prompt fault, and a more capable tier fixes neither. The counting endpoint above is how you stay inside the budget. A long supermarket chat that fails before any reply needs a shorter history, not a stronger model.

Sampling, Non-Determinism, and the Dial That Is Now Gone

The model does not always pick the single most likely next token. It samples from a distribution, and temperature widens or narrows that distribution. The glossary says: "Lower temperatures result in more conservative and deterministic outputs that stick to the most probable phrasing and answers". The word deterministic there is comparative. The entry's next paragraph says so. "Even with temperature set to 0, the results will not be fully deterministic and identical inputs may produce different outputs across API calls". The entry adds: "This applies both to Anthropic's first-party inference service and to inference through third-party cloud providers".

Temperature zero buys consistency, not reproducibility. No setting buys byte-identical output. Two shoppers asking the same question can get differently worded replies. Never build a component whose correctness requires two identical calls to return identical bytes. Constrain the shape instead, with a worked example and an explicit output contract. Validate the response at the boundary. Store an artefact you need to be stable rather than regenerating it and comparing. The supermarket stores its allergen statement once.

Model Options: Which Lever for Which Problem

Four options move a request along the same reasoning, latency and cost axis. They are not interchangeable.

Extended thinking is the original manual form: thinking turned on with an explicit token budget. It is the older interface: the legacy Claude Haiku 4.5 still uses it, and the newest models reject it.

Adaptive thinking replaces it with a delegation. Setting thinking: {type: "adaptive"} lets Claude "decide when and how deeply to think based on the request". As of 14 October 2026, on Claude Opus 5.5, Opus 5, Sonnet 5.5, Sonnet 5, Haiku 5.5, Fable 5.1, Fable 5 and the Mythos models, "thinking is already on and needs no configuration". Reach for it when a hard problem is being answered shallowly, such as a refund dispute that gets a one-line reply.

As of September 2026 the thinking configuration also takes a display field, which defaults to "omitted" on those models. Thinking blocks then arrive with an empty thinking field, still billed and passed back as before. Opt in with display: "summarized" and you get a readable summary, never the raw chain of thought. To see the reasoning, read the thinking blocks rather than asking for it in the answer. On Claude Fable 5.1, Opus 5.5, Opus 5, Sonnet 5.5 and Fable 5, a request that tries to pull the internal reasoning into the response text can be refused. The refusal carries stop_details.category: "reasoning_extraction", and that category is billed even when the refusal arrives before any output.

Effort is the dial you turn most often, and it is the workhorse:

kiln_note_prompt = "Summarise this kiln note for the day book: burners cycled twice overnight."

response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=8192,
    messages=[{"role": "user", "content": kiln_note_prompt}],
    output_config={"effort": "low"},
)

const kilnNotePrompt = "Summarise this kiln note for the day book: burners cycled twice overnight.";

const response = await client.messages.create({
  model: "claude-opus-5-5",
  max_tokens: 8192,
  messages: [{ role: "user", content: kilnNotePrompt }],
  output_config: { effort: "low" }
});

Levels run low, medium, high, xhigh, max, and the default depends on the model. As of 14 October 2026, "Most Claude models default to high effort", while "Claude Opus 5.5 and Claude Haiku 5.5 default to medium". "Setting effort to the model's default ("medium" on Claude Opus 5.5 and Claude Haiku 5.5, "high" on other models) produces exactly the same behavior as omitting the effort parameter entirely". Omit it on Claude Opus 5.5 and the request runs one level lower than it did on Claude Opus 5.

Two properties make effort the first lever to try. It "applies to every output token", so "it works whether or not thinking is enabled". It covers tool calls and their arguments too. So "Lower effort also means fewer and terser tool calls", not merely shorter prose. It is "a behavioral signal, not a strict token budget".

Fast mode is the only one of the four that does not trade quality. It "runs the same model with a faster inference configuration. There is no change to intelligence or capabilities". It delivers a documented multiple of the standard output rate at premium pricing, behind the fast-mode-2026-02-01 beta header with speed: "fast". Its gain is in output tokens per second, "not time to first token (TTFT)". So it shows on long streamed answers and barely registers on short ones. Switching between fast and standard speed invalidates the prompt cache. As of September 2026 it is a research preview limited to Claude Opus 5.5, Claude Opus 5 and Claude Opus 4.8 on the Claude API. The current throughput multiplier sits on the domain quick reference.

The selection rule is three sentences long. A hard problem answered shallowly wants more reasoning. A routine problem that costs too much wants less effort. An answer that is already correct but arrives too slowly for a person watching wants faster token delivery. That means streaming first and fast mode where it exists, not a different model. So the supermarket uses thinking for refund disputes, lower effort for opening hours, and streaming for slow correct replies.

Zero-Shot, Single-Shot, Multi-Shot

Zero-shot is an instruction with no demonstration. Single-shot adds one worked input-output pair. Multi-shot adds several. The current documentation calls the same family "few-shot or multishot prompting". It states the durable claim: "Examples are one of the most reliable ways to steer Claude's output format, tone, and structure. A few well-crafted examples... improve accuracy and consistency".

An example demonstrates a format and a decision boundary at once, in the same representation the model will generate in. A description of a format is a second-order instruction the model has to translate first. Two real outputs shown side by side beat a sentence describing what an output should look like. So the supermarket shows two past refund replies.

Examples should be "Relevant: Mirror your actual use case closely". They should be "Diverse: Cover edge cases and vary enough that Claude doesn't pick up unintended patterns". And they should be "Structured: Wrap examples in <example> tags... so Claude can distinguish them from instructions". The documentation adds: "Include 3-5 examples for best results". Diversity is the test candidates underrate. A set of examples that all share an incidental property teaches that property. If every example reply confirms an item is in stock, the model learns that replies never report a shortage.

Reach for examples when the failure is one of form or of boundary: inconsistent structure, unclear classification edges, or drifting tone. Do not reach for them when the failure is one of budget (a request that will not fit) or of fact (something the model cannot know). Examples fix the supermarket's drifting reply layout. They cannot supply a price that is in none of its inputs. Examples cost input tokens on every call, which is why they interact with 5.4. A stable example block at the front of a prompt is the kind of prefix prompt caching exists for.

In the exam

Domain 5 is named for model selection, but that is its smallest sub-skill. Two-thirds of the marks sit in the fundamentals underneath it: how the model produces text, and what the layer beneath your call is doing. This lesson owns the first half. Most of it is a property of the model rather than of a release, so what you learn about the mechanism will not go stale. Don't rely on a tokens-per-word ratio.

The exam keys all three consequences of next-token generation. Under a latency constraint, a candidate who confuses input length with output length reaches for the wrong lever. The guide names the four model options together. The durable knowledge is which one addresses which complaint, as in the supermarket's three cases. Writing and iterating the prompts themselves is Domain 6.

Dated Facts

Where a fact is dated, this lesson says so. Every parameter name, level list and model list under Model Options is dated and lives on the domain quick reference.

Exam Traps

Practice Scenario

Build Exercise

Sources

• Glossary — Claude Docs, tokens, latency, TTFT, temperature and the non-determinism statement
• Context windows — Claude Docs, the shared budget, cached prefixes, thinking tokens and both overflow behaviours
• Token counting — Claude Docs, the endpoint, its estimate status, its rate limits and the tokenizer change
• Thinking — Claude Docs, adaptive thinking, the display setting and the sampling-parameter restriction by model
• Refusals and fallback — Claude Docs, the refusal categories including reasoning_extraction, and which ones are billed
• Effort — Claude Docs, the levels, the per-model default and what effort affects
• Fast mode (research preview) — Claude Docs, speed, the beta header, OTPS versus TTFT and cache invalidation
• Model deprecations — Claude Docs, the API parameter deprecations table
• Prompting best practices — Claude Docs, examples as a steering technique and the relevance, diversity and structure tests
• Claude Certified Developer Foundations Exam Guide — Domain 5, LLM Fundamentals — Anthropic