Quick Reference: Domain 4 — Prompt Engineering & Structured Output

Concrete code examples beat prose descriptions. Instead of writing "use clear variable names", show a before/after code snippet.

Severity calibration: Without examples of what constitutes "critical" vs. "minor", the model treats all issues as equally important. Provide 2–3 calibration examples showing severity levels with specific code samples.

Structure a system prompt as:

1. Role and context (who the model is, what it is doing)
2. Rules and constraints (what it must/must not do)
3. Output format specification
4. Calibration examples (severity, tone, detail level)

Key rule: System prompts are the most cost-effective way to control behaviour — they are cached and reused across requests via prompt caching.

The exam's preferred pattern: Define a tool whose sole purpose is to structure output (e.g., extract_entities), then force it with tool_choice: { type: "tool", name: "extract_entities" }. This guarantees the response matches your schema.

Why tool_use over prompt-based JSON:

• Schema is validated by the API, not by post-processing
• No risk of malformed JSON, missing fields, or extra fields
• Works with streaming (structured chunks)

• Make fields optional or nullable when data may be missing. If a field is required but the data is absent, the model will fabricate a value to satisfy the schema.
• Use enum for constrained choices to prevent freeform values.
• Keep schemas flat where possible — deeply nested schemas increase extraction errors.
• Include description fields on each property — these guide the model on what to extract.

Fabrication prevention rule: If a field might not have a real value, make it nullable: true or required: false. This gives the model permission to return null instead of inventing data.

When to use few-shot: When instructions alone produce inconsistent output. Few-shot examples demonstrate the exact format and reasoning you expect.

Best practices:

• 2–4 examples is the sweet spot. More adds context cost; fewer may not cover edge cases.
• Include reasoning in examples, not just input → output. Show why the answer is what it is.
• Cover edge cases — at least one example should demonstrate boundary behaviour.
• Place examples in the system prompt for caching benefits, or in the user message for per-request variation.

Diminishing returns: Going from 0 to 2 examples has the largest impact. Going from 4 to 8 rarely improves quality and costs more context.

Multi-step pipelines where each step is a focused prompt with a single responsibility.

Pattern: Step 1 (extract) → Step 2 (validate) → Step 3 (format) → Step 4 (synthesise)

Advantages:

• Each step has a smaller, focused prompt — higher accuracy per step
• Intermediate results can be validated before proceeding
• Individual steps can be retried without rerunning the entire chain
• Different models can be used for different steps (cost optimisation)

When to chain vs. single prompt: Chain when the task has distinct phases with different requirements. Use a single prompt when the task is cohesive and the output format is straightforward.

When output fails validation, send back: original prompt + failed output + specific validation error.

The model sees what it produced, what was wrong, and can correct specifically. Never just say "try again" — always include the specific error.

When to retry vs. escalate:

• Retry: Validation error, format mismatch, missing field — the model can fix it
• Escalate: Repeated failures (>2 retries), confidence below threshold, fundamentally wrong interpretation

Key exam point: Batch API is for throughput and cost, not speed. If the question mentions "real-time" or "user-facing", Batch API is wrong.

A model cannot effectively review its own output in the same session. It retains the reasoning that produced the original output and is biased toward confirming it.

Fix: Use a separate model instance (new conversation, no shared history) for review. The reviewing instance sees only the output, not the reasoning that produced it.

For large inputs: Use per-file passes (analyse each file independently) plus a cross-file integration pass (synthesise findings). This is the "attention dilution" mitigation — processing everything in one pass causes the model to miss details.