A Claims Agent Under a Token Budget

The question and what it scores

The interviewer says: "Design an agent that processes insurance claims using RAG, and keep the token cost under control." The second clause is the real question. Anyone can sketch an agent with five tools. Few can say what it costs per claim before building it.

They are scoring four things:

• Design first. You can say what goes into each model call and why, before you reach for optimisations.
• Cost from first principles. You compute tokens in and out per step, multiply by steps and price, and know why the input side dominates.
• The levers in order of payoff, with the quality risk of each one stated.
• Measurement. Cost per claim is a metric you track, next to an accuracy metric, so a saving never hides a regression.

The design before the optimisation

Chapter 04 put claims intake in a workflow with a bounded agent inside it. This chapter designs the part that runs after the facts are collected: the claims agent that reads the evidence, finds the right procedure, checks coverage and routes the claim. About 6,000 new claims arrive a day.

Inputs. The first notice of loss text (about 800 tokens), up to three photos, and the policy record. Tools. Five of them:

• get_policy returns the policy record. Raw, it is a large JSON document of about 3,000 tokens.
• search_procedures is the RAG tool. It searches Harbourline's claims procedures and returns the top passages (chapter 01 covers when this retrieval goes wrong).
• check_coverage calls the rules engine with the peril, the date and the facts. It is deterministic code, and the answer is a typed verdict.
• fraud_flags returns the score and the reasons from the existing fraud model.
• route_claim places the claim in an adjuster queue.

One point of design matters for cost and for risk. The agent prepares a coverage recommendation and routes. A denial, or a payout above a threshold the business sets, goes to a human adjuster. A wrong coverage decision is a regulatory issue, so the agent's job is a complete, correct packet, not the final word.

The tool schemas also cost tokens. Anthropic adds a fixed tool-use system prompt of about 286 tokens for current models when tools are present, and your own tool names, descriptions and schemas are counted on top, as of November 2026.

Why the input side dominates

The API is stateless. Every call re-sends the tool definitions, the system prompt and the full history so far. So a piece of text that enters at step k of n is billed on every call from k to n, which is n − k + 1 times. Output tokens are billed once, at a higher rate per token (five times the input rate on Sonnet 5.5, as of November 2026), but there are far fewer of them.

Take a naive version with seven model calls. The 6,000-token prefix is sent seven times, which is 42,000 tokens. The 5,300-token first message, with its photos, is also sent seven times, which is 37,100. The raw policy record (3,000 tokens) is sent six times, and two search results of 4,800 tokens each are sent five and two times. Add those up and about nine of every ten input tokens are repeats of three things: the prefix, the first message and the large tool results. The model writes only about 1,850 tokens in the whole run, but reads about 142,000.

Photos deserve a note, because they are the largest single item in the first message. Claude reads an image in 28 by 28 pixel patches, so a picture costs roughly its width divided by 28, times its height divided by 28, tokens, capped by a per-model limit. A 1,000 by 1,000 pixel photo costs about 1,300 tokens. A large phone photo is downscaled to the model's cap, which is up to 4,784 tokens on the high-resolution tier (Claude 4.7 and later) and about 1,560 on the standard tier (Anthropic vision documentation, November 2026). Photos resized to about 1,090 pixels square cost about 1,500 tokens each; the three-photo first message in this model assumes that size.

A per-claim cost model you can run

The script below prices a claim as the sum over calls of input tokens times the input rate, plus output tokens times the output rate. With caching on, the prefix and history are billed at the cache-read rate and only the new tail is billed at the cache-write rate. The token counts per step are assumptions for Harbourline, not measurements; you would replace them with counts from your own traces. The prices are Anthropic's list prices as of November 2026.

"""Per-claim token cost model. Prices in USD per million tokens, as of November 2026."""
PRICE = {                          # input, 5-minute cache write, cache read, output
    "sonnet-5.5": (2.00, 2.50, 0.10, 10.00),
    "haiku-5.5":  (0.10, 0.125, 0.01, 0.50),   # prompts up to 100k tokens
}

def claim_cost(model, prefix, steps, cached):
    """steps = [(new_input_tokens, output_tokens), ...]. Every call re-sends all history."""
    p_in, p_write, p_read, p_out = PRICE[model]
    history = usd = 0
    for new_in, out in steps:
        if cached:   # prefix and history come from cache; only the new tail is written
            usd += (prefix + history) * p_read + new_in * p_write
        else:
            usd += (prefix + history + new_in) * p_in
        usd += out * p_out
        history += new_in + out
    return usd / 1e6

PREFIX = 6_000   # system prompt, tool schemas, coverage rules
# (new input tokens, output tokens) for each model call
NAIVE = [(5300, 300), (3000, 150), (4800, 200), (1200, 150), (800, 150), (4800, 200), (200, 700)]
LEAN  = [(5300, 300), (500, 150), (1800, 200), (400, 150), (200, 100), (150, 600)]
SLIM  = [(1200, 300), (500, 150), (1800, 200), (400, 150), (200, 100), (150, 600)]
PHOTO_PASS = [(5300, 400)]         # one small-model call reads the photos once

def row(label, usd, claims_per_day=6000):
    print(f"{label:38s} ${usd:.4f}/claim  ${usd*claims_per_day:>6,.0f}/day  ${usd*claims_per_day*30:>7,.0f}/month")

row("1 naive loop, no cache", claim_cost("sonnet-5.5", PREFIX, NAIVE, False))
row("2 + prompt caching", claim_cost("sonnet-5.5", PREFIX, NAIVE, True))
row("3 + trimmed tool results, one search", claim_cost("sonnet-5.5", PREFIX, LEAN, True))
big = claim_cost("sonnet-5.5", PREFIX, SLIM, True)
small = claim_cost("haiku-5.5", PREFIX, SLIM, True)
photo = claim_cost("haiku-5.5", 1_500, PHOTO_PASS, False)
row("4 + photos read once, summary kept", photo + big)
SHARE = 0.6    # assumption: share of claims the small model can finish. Measure it.
row("5 + 60% of loops on the small model", photo + SHARE * small + (1 - SHARE) * big)

Running it prints the table below. The first row is the baseline and each later row adds one lever to the row above it.

The whole stack cuts the cost by a factor of about twenty. Caching alone gives the first 73%, because it acts on the biggest repeated block with no change to behaviour. Notice also that output tokens, which were 6% of the naive cost, are about 35% of the cost in row 3. Once the input side is cheap, output becomes visible, and that is when a max_tokens cap and terse structured outputs start to matter.

At 6,000 claims a day the naive design costs about $54,600 a month. At 60 claims a day it would cost $546, and none of this work would be worth the time. State that scale argument in the interview. It shows you optimise for the bill that exists.

The levers, in order of payoff

1. Cache the prefix and the history. Anthropic's prompt caching stores the prompt prefix, ordered tools, then system, then messages. A later call that begins with an identical prefix reads it from cache. As of November 2026, a read costs 0.05 times the base input price on Sonnet 5.5 ($0.10 against $2 per million tokens), a five-minute write costs 1.25 times, and a one-hour write costs 2 times. Three conditions decide whether it works. The prefix must be at least 512 tokens on Sonnet 5.5, which 6,000 is. The prefix must match exactly, so a timestamp or claim id in the system prompt, placed before the breakpoint, defeats it. And any change to the tool definitions invalidates everything after them, so editing a tool description in production resets the cache for every claim. The default lifetime is five minutes and each read refreshes it. At 6,000 claims a day, claims arrive on average about every 15 seconds, so the prefix stays warm in the day. For a quiet overnight spell, either accept a re-write or pay the one-hour write price. Check the cache_read_input_tokens field in the usage data; if it is zero, the prompt was not cached.

2. Make tools return less. The raw policy record has hundreds of fields and coverage needs a dozen. Code, not the model, should project the fields the next step needs, so the tool returns 500 tokens and not 3,000. Anthropic's context-engineering guidance calls clearing stale tool results one of the lightest and safest forms of compaction. The risk is dropping a field a later step needs, so keep an id the agent can use to fetch the full record on demand.

3. Retrieve once, at the step that needs it. The naive loop searched the procedures twice, with eight passages each time, because it re-searched on every turn. Search after the facts and peril are known, ask for the top three passages, and keep them. Smaller k only works if ranking is good, so this lever depends on the reranking work in chapter 03. Measure recall at three, not at ten, before you cut.

4. Read the photos once. A small model reads the three photos in one call and writes a 400-token structured damage summary. The main loop carries the summary, not the pixels, so the images are not billed on six more calls. Add a view_photo tool that fetches one image on demand for the rare step that needs the detail. The cost is lost detail in the summary, so test it on the claims where damage severity decides the route.

5. Route by difficulty. Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens (for prompts up to 100,000 tokens, as of November 2026), against $2 and $10 for Sonnet 5.5. That is a factor of twenty. Run the loop on the small model for claims that look simple, and escalate to the larger one when check_coverage returns "ambiguous", the fraud score is high or the small model reports low confidence. The 60% share in the table is an assumption. You find the real share by running both models on the eval set and counting how often the small model reaches the same verdict. The saving here is about $100 a day, and one wrongly auto-routed claim that becomes a complaint can cost more than that. So the router needs an escalate-when-in-doubt rule and its own eval slice, not a hunch.

6. Cap everything. Set max_tokens on every call, a limit of model calls per claim, and a total token budget per claim, for example 120,000 input tokens. When any limit hits, the claim goes to a human with the partial packet. Chapter 14 shows the guard. The cap also bounds the worst case, which matters because the average hides the p99 claim that loops.

7. Cache embeddings. Embed each procedure passage once, at index time. Cache query embeddings only if a retry reruns the same search. Embedding calls cost far less than generation here, so this is housekeeping, not a lever. Say so, and move on.

Two things that look like levers are not. The Batch API gives a 50% discount on input and output, but it returns results asynchronously, and a loop that needs six sequential calls per claim does not fit. Use it for the nightly classification and summary jobs in chapter 09. And a very long context is not a substitute for retrieval: a 300-page policy file at about 500 tokens a page is roughly 150,000 tokens, and even at the cache-read price that is $0.015 for every call, which is $0.105 over seven calls, more than the whole lean pipeline costs.

Cost per claim as a first-class metric

A bill that arrives monthly cannot tell you which step grew. Log one record per model call: claim id, step name, model, input tokens, cache-read tokens, cache-write tokens, output tokens and the computed cost. Sum by claim id to get cost per claim, and sum by step name to see where it goes. Then track:

• Cost per claim at p50, p95 and p99. The mean hides the looping claim. A p99 ten times the p50 is a sign of an uncapped path.
• Cost by step. A new tool that returns twice the tokens shows up as one step's share jumping, the day it ships.
• Cache hit rate, as cache-read tokens over all input tokens. A drop after a deploy usually means someone edited the prefix.
• Calls per claim. A rising average is the cheapest early warning of a loop.
• Accuracy on the eval set, beside every cost number. The release gate is "cost fell and routing accuracy did not", as in chapter 07. For the broader view of latency and cost together, see chapter 10.

Interview probes

Answer each aloud before opening it.

✓ Check yourself

Answer from memory before opening each one.

1. An agent makes eight model calls. A 4,000-token tool result arrives on call 2 and nothing removes it. About how many input tokens does that one result cost across the run, and what single change cuts it most?

   Show answer

   It is sent on calls 2 to 8, so seven times: 28,000 input tokens. Having code project only the needed fields (or clearing the result after it is used) cuts it most, because the cost scales with how long the text stays in the history. See why the input side dominates.

2. You enable prompt caching with a 6,000-token system prompt, but the usage data shows zero cache-read tokens on every call. The prompt starts with "Today is {date}, you are helping claim {id}". What is wrong, and what is the fix?

   Show answer

   The per-request date and claim id sit at the start of the prefix, so no two calls share an identical prefix and every call is a miss. Move the static instructions first, put the breakpoint after them, and put the date and id in the user message after the breakpoint. See lever 1.

3. A teammate proposes moving every claim to the small model, since it is twenty times cheaper and "most claims are easy". Using the numbers in this chapter, what is the most the change can save per day, and what would you require before agreeing?

   Show answer

   Against the row-4 design the ceiling is about $190 a day, and the realistic saving is less, since the large model is still needed for hard claims. Require an eval run that shows the small model's coverage verdicts match on every slice, an escalate-on-doubt rule, and a shadow period. One extra complaint can cost more than the day's saving. See lever 5.

4. After caching and trimming, a dashboard shows output tokens are now about a third of the per-claim cost. The team wants to cut input further. Argue for or against.

   Show answer

   Output is billed at five times the input rate, so at that mix the cheapest remaining saving may be on the output side: a lower max_tokens, a structured verdict instead of prose, and a shorter adjuster summary. Check that the summary's length is not carrying information adjusters use. Input cuts now risk accuracy for smaller returns. See the cost model.

5. Why can the Batch API not be your main cost lever for the claims agent, even though it halves the price?

   Show answer

   Each step of the loop depends on the previous step's result, so the calls are sequential, and a batch returns asynchronously, so the delay would multiply across the steps. It fits independent, deferrable work such as nightly summaries. See the closing note on levers.

Takeaway

• Agent cost is context times calls: every call re-sends the prefix and the history, so text that stays in the history is billed again on each later call.
• In the Harbourline model about nine of every ten input tokens are repeats of the system prompt, the first message and large tool results.
• Prompt caching comes first, since it acts on the biggest repeated block without changing behaviour, and it only works with an identical prefix, the minimum length and traffic frequent enough to keep the entry warm.
• Trim tool results in code, retrieve once at the step that needs it, and read photos once, each of which removes tokens from every later call.
• Route to a smaller model only with an eval slice, an escalate-when-in-doubt rule and a shadow period, because the saving is small next to the cost of one wrong coverage decision.
• Log cost per call, aggregate per claim and per step, track p95 and p99, and never report a cost change without the accuracy change beside it.

Next, the other agent in the programme, the one customers talk to directly. → A Customer-Service Agent

Further reading

• Prompt caching: Anthropic documentation. Prefix order, breakpoints, minimum lengths, lifetimes, what invalidates the cache and the usage fields.
• Pricing: Anthropic documentation. Per-model input, cache and output prices and the Batch API discount; check it again before quoting any number.
• Vision: Anthropic documentation. The patch-based image token formula, per-model caps and the size table.
• Effective context engineering for AI agents: Anthropic engineering, September 2025. Tool-result clearing, compaction and just-in-time retrieval.