A Customer-Service Agent

The question and what it scores

The interviewer says: "Design a customer-service agent for Harbourline's web chat." This is the broadest question in the guide, and the interviewer expects you to run it as a conversation: ask what is in scope, draw the system, then defend it under push-back.

They are scoring four things:

• Scope before boxes. You ask what the agent must not do, and you can name the non-goals.
• A permission model for tools that does not depend on the prompt. Reads, writes, and who authorises each.
• What happens when it fails: escalation, injection, wrong answers and PII, treated as design inputs.
• How you would know it works, and how you would put it in front of 40,000 sessions a day without finding out the hard way.

Requirements, non-goals and channels

Start with what it does. At Harbourline, about 40,000 chat sessions a day, the agent answers questions about a customer's policy using the help-centre documents, reports the status of a payment, takes a request to change an address, and hands over to a human when it should. Those four jobs cover most of the volume and carry modest risk.

Then state the non-goals, because they are where the risk sits. The agent does not decide coverage, does not discuss the merits of a disputed claim, does not give legal advice, and does not cancel a policy or promise a payout. Each of those goes to a human with the context attached. The reason is the cost of a wrong answer. A wrong deductible quoted in chat is a complaint. A wrong coverage statement can bind the insurer or breach a regulator's expectations. Courts and tribunals have already held a company responsible for what its chatbot told a customer; the widely reported 2024 Air Canada bereavement-fare ruling in British Columbia is the usual example, and the lesson interviewers take from it is that "the bot said it" is not a defence.

Channels matter more than they seem. Web chat is the first one, but the phone line is moving to voice (chapter 12) and email is next. Build the agent core once and put thin adapters around it. The adapters handle rendering (buttons in chat, spoken confirmation on the phone) and nothing else. The tool set and the policy layer stay shared, so a safety rule is written once.

Authentication shapes everything below. An anonymous visitor can search the help centre and nothing else. A logged-in customer gets account tools. The tools offered depend on the auth level of the session, set by code before the model sees the request. A prompt that says "only help logged-in users" is a request, not a control.

The shape of the system

The model never touches a backend directly. Every tool call passes through a policy layer written in ordinary code, and that layer is where authorisation, schema checks, confirmation and the audit log live.

The five tools are deliberately few. Every tool adds its description to every call's input, and more tools make the model's choice among them harder. If two tools are always called together, merge them. If the agent needs a policy summary and a payment status in most sessions, one get_account_summary call, or a summary your code prefetches at session start, is faster and cheaper than two sequential calls.

Tools and their permission levels

Two mechanisms carry the design. The first is account scope from the session. If get_payment_status took a policy id argument, a customer could type another person's number and the model would pass it on. With no such argument, the tool can only read the account that the session proved. This closes the whole class of "look up someone else's data" attacks without asking the model to behave.

The second is the staged write. An address change is a common step in account takeover, so it should never complete on the model's say-so. The tool stages the change and the application draws a confirmation with the old and new address and a button. The click calls a separate endpoint that the model has no tool for. The sketch shows the pattern:

# Sketch: a write tool the model can request but cannot complete.
# validate_address, apply_change and session.ui are assumed helpers.
import secrets, time

PENDING = {}   # token -> (session_id, action, args, expires_at); a table in production

def request_address_change(session, new_address):
    """Model-callable tool. Stages the change and shows a button. Never applies it."""
    args = validate_address(new_address)               # bad input raises a tool error
    token = secrets.token_urlsafe(16)
    PENDING[token] = (session.id, "update_address", args, time.time() + 300)
    session.ui.show_confirmation(                      # drawn by the app, not by the model
        title="Change your address?",
        old=session.account.address, new=args, token=token)
    return {"status": "awaiting_customer_confirmation"}

def confirm(session, token):
    """Endpoint behind the Confirm button. No tool the model holds can reach it."""
    sid, action, args, expires = PENDING.pop(token, (None, None, None, 0))
    if sid != session.id or time.time() > expires:
        raise PermissionError("unknown, expired or foreign token")
    apply_change(session.account.id, action, args)     # writes an audit row too

def get_payment_status(session):
    """Read tool. Takes no policy id: the session fixes which account is in scope."""
    return payments_api.latest(session.account.id)

Memory: what to keep, and why so little

There are two kinds. Session memory is the conversation itself. It grows with every turn and every tool result, and each call re-sends it (chapter 05 explains the cost). Trim it: clear stale tool results, and summarise older turns once the chat runs long. Anthropic's context-engineering guidance names both techniques, and warns that over-aggressive summarising can drop small details that matter later.

Long-term memory is what survives between sessions. My default for Harbourline is a narrow version: code, not the model, writes structured, customer-visible facts from tool outcomes, such as "preferred contact channel" or "open ticket reference", with a retention period and deletion tied to the account. I would not give the model a free-form memory it writes itself, for three reasons. Customers say personal things in chat, so a free-form store becomes a store of personal data. Text written into memory is read back in a later session as if it were trusted, so an injected instruction saved once is replayed forever. And a wrong "fact" the model recorded is hard to find and correct. Anthropic's memory tool leaves storage, size limits and expiry to your application, and notes the model usually declines to save sensitive data but that validation is needed for stronger guarantees. That is a good sign the design belongs in your code.

Where this flips: a low-risk preference store, such as "send documents by email", written through a typed tool with a schema, is fine and useful. The rule is that memory has a schema, an owner and an expiry date.

Escalation and the handover

The agent hands over when any of these holds: the customer asks for a person; the topic is a non-goal; the same intent is unresolved after two attempts; a tool fails twice; retrieval finds nothing relevant, so the honest answer is "I don't know"; or the language signals a complaint or distress. Put the triggers in code where you can, such as the retry counter and the retrieval score, and in the prompt where you cannot, such as detecting distress.

The test of a good handover is that the customer does not repeat themselves. The packet that appears on the human agent's screen holds:

• who the customer is and how they were authenticated;
• the intent in one line, then what the agent already did, as tool calls and results with times;
• the last three customer messages verbatim;
• what is unresolved, and a suggested next step marked as a suggestion.

If no human is online, say so, offer a callback and create the ticket. Do not let the agent imply that someone is about to join. Be careful with the metric as well: if the team is rewarded for containment alone, the cheapest way to raise it is to bury the escalation option. The next sections return to that.

Safety: injection, personal data and refusals

Prompt injection. OWASP lists prompt injection first in its Top 10 for LLM applications (2025 edition) and splits it in two. Direct injection is the customer typing instructions at the model. Indirect injection is instructions hidden in content the model reads: a pasted email, a note field on a claim, a help-centre page. OWASP says it is unclear whether injection can be fully prevented, and names three mitigations: least privilege, human approval for privileged actions, and separating untrusted content. The design above is exactly those three. Tools are scoped to the session's own account, so even a fully hijacked model can only touch the attacker's own data. Writes need a click. Customer text and tool results are passed as clearly marked data, not as instructions. And the toolbox has no tool that can send data out to an arbitrary address or fetch an arbitrary URL, since that is the path an injected instruction would use to leak.

Personal data. Return only the fields the answer needs, so a tool result is a short typed record and not the whole customer row. Redact card and identity numbers before text reaches the model or the logs. Remember that the transcript store is itself a store of personal data, with its own retention and access rules. Where a rule depends on law, such as limits on solely automated decisions with significant effects, ask legal. I am not giving a legal reading here.

Refusals. A refusal that ends in "I can't help with that" is a failure. For each non-goal, the response states what it can't do, why in a sentence, and hands over. For numbers like a deductible, the rule is that the agent quotes only what a tool or a cited help-centre passage returned. It never quotes from the model's own memory. That one rule removes the most common source of wrong figures.

Evals and rollout

Build the offline set from sampled production transcripts, sliced by intent, plus synthetic cases for rare ones and red-team cases for injection (chapter 07 covers building and sizing it). The metrics, and what each hides:

• Resolution rate. The task was completed, checked against system state (the address changed, the status was correct), not against the agent's own claim that it was done.
• Containment. No human was needed. It is meaningless alone, since a bot that blocks escalation has high containment. Report it with resolution and with the share of customers who contact again within seven days.
• Hallucination rate. Statements not supported by a tool result or a cited passage, measured on a labelled sample. Track it per intent.
• Unsafe action attempts. A write without a confirmation. The target is zero, and any non-zero is a bug in the policy layer.
• Escalation quality. Did it escalate when it should have, and did it escalate when it did not need to. Both errors cost, in different ways.
• CSAT. The customer's rating. It is noisy, and only the people who answer the survey are counted.

For the agent's tool use under a simulated customer, τ-bench (Yao et al., 2024) is the public reference. It pairs a simulated user with an agent that has domain tools and policy rules, scores the final database state against a goal state, and introduced the pass^k measure of whether the agent succeeds on every one of k trials. When published, its authors reported that agents using function calling were inconsistent and that pass^8 in the retail domain was under 25%. Reliability across repeated runs, not a single good run, is the right bar for a customer-facing agent. Chapter 08 goes into trajectory evals.

Rollout goes in stages, with a gate and a kill switch at each one:

1. Shadow. The agent runs on live chats and its answers are logged, not shown. You compare against what the human agents did.
2. Staff and read-only intents. Employees first, then customers on the read-only intents (policy questions, payment status).
3. A small share of traffic, such as 1%, then 5%, 25%, 100%, adding the write intents last.

Sample size sets the pace. A 1% slice of 40,000 sessions is about 400 a day. With zero failures in n trials, the rate of three divided by n is the approximate 95% upper bound (the "rule of three"). After a week at 1%, about 2,800 sessions, a clean run bounds the failure rate near 0.1%. If your tolerance for a wrong deductible is one in ten thousand, you need more traffic or a better-targeted eval before you can claim it.

The trade-offs interviewers push on

Latency against tool count. Each tool round trip is a full model call plus the tool's own latency, so an answer that needs three sequential tools takes at least three model calls. Four fixes, in order: merge tools that are always used together; prefetch the account summary in code at session start, so the model begins with it; allow parallel tool calls when two lookups are independent; and stream the final answer so the customer sees words while the rest is produced. Each costs something. Merged tools return more than some questions need, and prefetching spends tokens on sessions that never ask about the account. Prefetch only if most sessions use it. Measure the p95 time to the first word, not the mean (chapter 10).

Containment against harm. Raising containment means the agent must answer more, and a more willing agent gives more wrong answers. The costs are not symmetric. A needless handover costs a few minutes of a human agent's time. A wrong deductible costs a complaint and may cost more. So set the thresholds by intent: push containment hard on payment status, where wrong answers are rare and cheap to correct, and escalate early on anything that touches coverage. If the interviewer asks which number you would put on the dashboard first, the answer is hallucination rate per intent, with containment beside it.

Where this design flips. If Harbourline had 400 sessions a day and three intents, a decision tree with a model for language understanding would beat an agent, and chapter 04 is the argument. The agent earns its cost where the volume is high and the long tail of questions is wide.

Interview probes

Answer each aloud before opening it.

✓ Check yourself

Answer from memory before opening each one.

1. A tool takes a policy_id argument and a system prompt says "only use the id of the logged-in customer". A tester gets another customer's data by typing that customer's number. Why did the prompt fail, and what is the structural fix?

   Show answer

   The model passes through whatever argument the conversation supplies, and an instruction in a prompt is a request that customer text can override. Remove the argument and let the policy layer read the account from the authenticated session. Then there is nothing for the model to get wrong. See tools and permission levels.

2. The customer types "yes, go ahead" in the chat after the agent proposes an address change, and the agent applies it. A reviewer calls this a vulnerability. What is the attack, and how does the staged write remove it?

   Show answer

   Text in the chat can be produced by an injected instruction or by anyone with access to the session, so "yes" in the chat proves nothing. With a staged write, the model can only request the change; applying it needs a click on a confirmation drawn by the application, tied to a token for that session. The model has no tool that reaches that endpoint. See the staged write.

3. A product owner wants the agent to "learn from every conversation" by writing its own notes to a memory store that it reads at the start of each session. List the risks you would raise and the design you would offer instead.

   Show answer

   Risks: personal data collects in an unplanned store, injected text persists and gets replayed as trusted, and wrong notes are hard to find. Offer typed, customer-visible fields written by code from tool results, each with an owner, a retention period and a deletion path. See memory.

4. A dashboard shows containment of 90% on the payment-status intent and 90% on the coverage-question intent. Which number worries you, and what would you check?

   Show answer

   The coverage figure. Coverage questions are a non-goal and should mostly escalate, so high containment suggests the agent is answering what it should hand over. Sample those chats and measure hallucination and unsupported-claim rate, and check whether the escalation trigger for that topic still works. See containment against harm.

5. You run 2,000 live sessions at 1% traffic with no unsafe action and no hallucinated figure. A director says that proves the failure rate is zero. What can you actually claim?

   Show answer

   Only an upper bound: about 3 in 2,000, or 0.15%, at 95% confidence. That is a ceiling, not a rate, and it applies to the mix of intents you sampled, not to rare ones. Keep widening in stages and keep the tripwires. See evals and rollout.

Takeaway

• Scope the agent by its non-goals first: coverage decisions, disputes, legal advice and cancellations go to a human, because a wrong answer there is costlier than a handover.
• Authorisation lives in a policy layer in code: the session fixes the account, arguments are validated, and the prompt is never the control.
• Writes are staged by the model and applied by a customer click on an endpoint the model cannot reach, so no injected text can complete an action.
• Keep long-term memory typed, owned and expiring, written by code from tool results, since free-form model memory stores personal data and replays injected text.
• A good handover means the customer never repeats themselves, and containment is only reported beside resolution, recontact and hallucination rate.
• Roll out in stages with a kill switch, and treat a clean sample as an upper bound on the failure rate, not as proof of zero.

Next, how to build the eval set that makes those rollout gates mean something. → The Eval Set and the Ship Decision

Further reading

• LLM01:2025 Prompt Injection: OWASP Top 10 for LLM Applications. Direct and indirect injection, and the least-privilege, human-approval and content-separation mitigations.
• τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains: Yao, Shinn, Razavi and Narasimhan, 2024. How to evaluate a tool-using agent against a simulated customer and why pass^k matters.
• Memory tool: Anthropic documentation. Client-side memory, and the security notes on sensitive data, size limits, expiry and path traversal.
• Effective context engineering for AI agents: Anthropic engineering, September 2025. Trimming session context: tool-result clearing, compaction, notes.
• Tool use with Claude: Anthropic documentation. The tool loop, tool_choice and parallel tool calls.