What You Need to Know
Decomposition splits one task into subtasks, each given its own model pass. A pipeline chains those subtasks as stages. Decompose a task that is too large, too varied, or too important to trust to one pass.
Take a car body shop. It wants an estimate from each repair job's file of notes and damage photos, and a monthly check of all its files.
Fixed vs Dynamic Decomposition
Fixed decomposition happens at design time: the architect breaks the task into known stages and builds a pipeline. The stages are known, so it is predictable and testable stage by stage. It suits structured repeatable work: document processing, review pipelines, report generation. Its workflow pattern is prompt chaining.
Dynamic decomposition happens at run time: the model discovers the task's structure as it works, generating subtasks from its findings. It suits open-ended work whose shape emerges during execution: investigation, research, exploratory analysis. Its workflow pattern is orchestrator-workers (objective 1.4).
The test: can the steps be written down in advance? If yes, fix them in code and gain predictability. If no, let the model decompose and gain adaptability, at the cost of predictability. The shop's estimate passes. Asking why repaired cars keep coming back fails: each finding sets the next step.
The Four Decomposition Axes
Choosing how to split is the real design decision, and the pattern follows from it.
- By stage: sequential phases with clear hand-offs, such as extract, analyse, draft, review. Use when later work depends on earlier output. The estimate splits this way, since pricing needs the damage list first.
- By section: independent units processed in parallel, such as per document, per file, per region. Use when units do not interact. The monthly check splits by job file.
- By perspective: the same input analysed through different lenses (security review plus performance review), or repeated for confidence via voting. Use when one viewpoint systematically misses what another catches.
- By specialism: subtasks needing different tools, prompts or models. Use when one configuration cannot serve all subtasks well.
Attention Dilution
Attention dilution happens when one pass covers too much material. Depth becomes inconsistent: detailed findings on some items, obvious misses on others, contradictory judgements on identical patterns in different places. A code review of fourteen files, thorough on three and blind on five, has a decomposition problem, not a model capability problem. The shop's single monthly pass might flag a cracked bracket on one job and miss it on another.
The fix is structural, with two mandatory halves. Neither suffices alone.
- Per-unit local passes: each file, section or document analysed alone. Every unit gets a full pass, so depth is consistent on every unit. Each job file gets its own pass. Without integration, these passes miss everything that lives between units.
- A separate integration pass: hunting the cross-unit issues (data flow between files, rules spanning contract sections) that local passes cannot see by construction, since each sees only its own unit. For the shop, that is one part billed on two jobs for the same car. Without local passes, a single integrated pass dilutes again.
The Bigger-Model Distractor
Exam stems showing inconsistent depth offer a larger model or bigger context window. Fit was never the failure; attention was. A model that holds 400 pages still dilutes across 400 pages. The answer is always local passes plus integration.
Interface Discipline
Between stages, intermediate outputs should be structured against a defined schema, not free prose. A schema has three effects.
- each stage becomes independently testable, because its output can be checked against the schema
- downstream stages stop re-interpreting prose
- attribution (source locations, clause references) travels through the pipeline as fields, such as the photo reference on each of the shop's damage records
Add acceptance gates between stages: programmatic checks that stop bad intermediate output propagating. Unchecked errors compound across stages. One hallucinated field in stage two becomes a confident conclusion in stage five. The shop's gate checks in code that every part number exists before pricing.
When Not to Decompose
Every split adds latency, cost and an interface to maintain, and errors propagate across stage boundaries. Decompose when one of these holds:
- quality measurably improves
- the task cannot fit one context
- stages need independent verification
A task that fits comfortably in one call and shows no dilution symptoms should stay there, like the shop's quote for one scratched door. A five-stage pipeline for it is pure overhead.
In the exam
The Professional exam tests decomposition as a design skill with named failure modes. Attention dilution is tested most heavily. It scores a five-stage pipeline for a one-call task as over-engineering.
Exam Traps
Processing a large corpus — many files, a 400-page contract — in one model pass
Attention dilution produces inconsistent depth and contradictory judgements on identical patterns. Per-unit local passes plus a cross-unit integration pass restore consistency. Window size does not fix attention.
Fixing inconsistent-depth symptoms with a larger model or bigger context window
Fit was never the failure. A model that holds the whole corpus still dilutes across it. The fix is structural decomposition, and the exam plants the bigger-model option as its favourite distractor here.
Decomposing into local passes only, with no integration pass
Cross-unit issues — data flow between files, rules spanning sections — are invisible to local passes by construction. The two-part fix is mandatory: local depth plus cross-unit integration.
Passing free-text summaries between pipeline stages
Downstream stages re-interpret prose, attribution is lost, and stages cannot be tested independently. Structured intermediate outputs with acceptance gates fix all three at once.
Practice Scenario
A compliance team must check 400-page loan agreements against 60 policy rules. A single-prompt approach misses violations inconsistently — some sections are analysed thoroughly, others barely. What should the architect design?
Build Exercise
Decompose a Document-Analysis Pipeline
What you'll learn
- How to choose a decomposition axis and defend it against alternatives
- How to design local passes plus an integration pass against attention dilution
- How to define structured intermediate outputs that preserve attribution
- How to write acceptance gates that stop error propagation between stages
- Choose a large-document analysis task: contract review, codebase audit, or long-report QA
Why: Attention dilution needs scale to appear, and the fix needs scale to matter. Pick something a single pass genuinely struggles with.
You should see: A one-line task statement including the corpus size and what a finding looks like.
- Pick the decomposition axis — stage, section, perspective, or specialism — and defend it against the other three in one sentence each
Why: The axis is the design decision; the pattern follows from it. Defending against alternatives is what makes the choice deliberate rather than habitual.
You should see: Four sentences: one justification and three rejections, each naming a property of the task.
- Define the intermediate output schema for the local passes, including source-location fields
Why: Structured interfaces make stages independently testable and carry attribution through the pipeline. Prose interfaces lose both.
You should see: A JSON schema for one finding: what was found, where (section, clause, or file and line), against which rule, at what confidence.
- Write the acceptance gate that sits between local passes and integration: what makes a local finding admissible
Why: Acceptance gates stop bad intermediate output propagating. Errors compound across stages if unchecked.
You should see: Two or three programmatic checks: schema validity, source location resolves to real text, rule identifier exists.
- Specify the integration pass: which cross-unit issues it hunts and why local passes cannot catch them
Why: The integration pass is the half of the fix designs forget. Its scope must be explicit or it degenerates into a second diluted mega-pass.
You should see: A short list of cross-unit issue types — rules spanning sections, contradictions between sections, aggregate thresholds — each with a line on why it is invisible locally.
Sources
- Building Effective AI Agents — Anthropic
- How we built our multi-agent research system — Anthropic