Model Selection and Optimisation
4 build exercises to practise the concepts in this domain.
5.1 — Measure the Budget, Then Break It Two Ways
Intermediate
35 minutes
- How to measure tokens before and after a request, and why the two numbers differ
- How a tokenizer change invalidates a cost model across a migration
- How the two context-window overflow behaviours differ in symptom and in fix
- How to stabilise output shape without depending on byte-identical responses
5.2 — Read the Wire, Then Take the SDK Away and Give It Back
Intermediate
45 minutes
- How to read a raw Claude request and response, headers and error envelope included
- Which failures the SDK retries for you and which it raises immediately
- How timeouts and retries compose into worst-case wall clock
- Which transport sits at which layer between a browser, your server and Anthropic
- Why a reused client behaves differently from a per-request one under concurrency
5.3 — Assign the Tiers, Then Earn the Migration
Intermediate
40 minutes
- How to name the single constraint that decides a workload’s tier
- How to build an evaluation set with boolean pass checks rather than ratings
- How to compare candidates on cost per completed task across effort levels
- How to gate a migration on evidence instead of on release notes
- Why a retirement date belongs in a planning calendar
5.4 — Put a Price on One Workload, Then Change One Thing at a Time
Intermediate
45 minutes
- How to price the token counts in usage and find the dominant axis
- How per-feature attribution turns a claimed saving into a measured one
- How to verify a cache hit and diagnose a silent invalidation
- How the cache lifetime interacts with a bursty arrival pattern
- Why lowering max_tokens truncates answers instead of shortening them