Core Fundamentals

LLM orchestration — the mental model

An "AI agent" in production is rarely just one model call. It's an orchestrated loop:

1. Receive an input — a Slack message, a survey close event, an n8n trigger, an HRIS webhook.
2. Build a prompt — system prompt + retrieved context + the input.
3. Call the model — Claude with tools available.
4. Model proposes a tool call — "read Workday worker by id," "list Ashby candidates."
5. Your orchestrator executes the tool — calls the API, returns the result.
6. Result goes back to the model — model continues reasoning or proposes another tool.
7. Loop until done — model returns a final structured output.
8. Hand off to HITL or commit — present to a human for approval, or commit if low-risk.

Three observations that show up in interview:

• The orchestrator (n8n, your Python code, or both) controls the loop — not the model. The model proposes; the orchestrator disposes.
• Every tool call is an audit-log entry. You write the log on entry and exit of each tool.
• The final action (the Workday write, the EIB upload) is almost always a tool call you implement as a HITL request, not as a direct write.

Agentic primitives — the building blocks

You will be asked some version of "design X." Your job is to compose these primitives. There's no extra magic.

Prompts as code

Prompts are not a creative-writing exercise. They are versioned artifacts with tests. The structure that survives:

• System prompt — role, constraints, output schema. Stable, prompt-cached.
• Knowledge context — policy library chunks, Workday schema, exemplars. Prompt-cached.
• Task instruction — what to do this call. Varies.
• Input data — the worker record, the survey response, the ticket. Varies.
• Output contract — the JSON schema or the markdown shape expected back.

Anti-patterns: prompts that live inline in n8n with no version control; prompts that mix knowledge with task instructions; prompts that don't specify an output contract.

Tools / function calling

A tool is a typed function the model can call. Three rules:

1. Name it like a verb on a noun. get_worker, propose_job_change, list_open_reqs. The model picks tools by their names; clarity matters.
2. Document each parameter and return shape. The description goes to the model.
3. Validate inputs server-side. Never trust the model to send valid values.

tools = [
    {
        "name": "get_worker",
        "description": "Fetch a worker record from Workday by employee_id. Returns name, position, supervisory org, location, FTE, effective dates. Returns null if not found. Read-only.",
        "input_schema": {
            "type": "object",
            "properties": {
                "employee_id": {"type": "string", "description": "Workday worker ID, e.g. 'W-12345'"}
            },
            "required": ["employee_id"],
        },
    },
    {
        "name": "propose_job_change",
        "description": (
            "Propose a job change for a worker. Does NOT commit — drafts an EIB row "
            "and returns a preview for human approval. Use this for any change to "
            "job profile, supervisor, location, or FTE."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "employee_id": {"type": "string"},
                "effective_date": {"type": "string", "format": "date"},
                "new_job_profile": {"type": "string"},
                "new_supervisor_id": {"type": "string"},
                "new_location": {"type": "string"},
                "new_fte": {"type": "number"},
                "reason_code": {"type": "string"},
            },
            "required": ["employee_id", "effective_date", "reason_code"],
        },
    },
]

Notice how the propose_job_change description tells the model explicitly that this is non-committing. That's the HITL boundary, encoded at the tool boundary, in language the model can act on.

Structured output

For any agent whose output is consumed by code or rendered in a HITL UI, you want JSON, not prose. Two ways with Claude:

• Tool-use for output — define a tool like submit_final_plan and require the model to call it. This gives you typed JSON.
• Prefill — start the assistant turn with { and instruct in the system prompt to return JSON only.

Always validate with Pydantic / JSON Schema before acting:

from pydantic import BaseModel, Field
from typing import Literal

class ProposedJobChange(BaseModel):
    employee_id: str = Field(..., pattern=r"^W-\d+$")
    effective_date: str  # ISO date
    new_job_profile: str | None = None
    new_supervisor_id: str | None = None
    new_fte: float | None = Field(None, ge=0, le=1.0)
    reason_code: Literal["PROMOTION", "LATERAL", "REORG", "FTE_CHANGE"]
    rationale: str

# After Claude returns:
plan = ProposedJobChange.model_validate_json(raw_json)
# If invalid, do not present to the human approver. Retry or fail cleanly.

Retries & backoff

Two categories of failure, two policies:

• Transient (5xx, rate limit, timeout) — retry with exponential backoff + jitter. 3-5 attempts. Most SDKs include this.
• Logical (model returned wrong shape, tool validation failed) — retry once with a corrective message ("your previous output didn't match schema X; here's what was wrong"). After one retry, fail clean.

What never retries automatically: anything that touches Workday writes, payroll, or has already had partial side effects. See 08-error-handling.

import time, random
from anthropic import Anthropic, APIStatusError, APITimeoutError

client = Anthropic()

def call_with_retry(messages, system, tools, max_attempts=4):
    for attempt in range(max_attempts):
        try:
            return client.messages.create(
                model="claude-sonnet-5-5",
                max_tokens=4096,
                system=system,
                messages=messages,
                tools=tools,
            )
        except (APITimeoutError, APIStatusError) as e:
            if attempt == max_attempts - 1:
                raise
            sleep = (2 ** attempt) + random.random()
            time.sleep(sleep)

Idempotency — the most underrated discipline in HR automation

Every action your agent could ever commit must be safely retriable. In HR this is non-negotiable because:

• An idempotency failure on a hire can create a duplicate worker record
• An idempotency failure on a comp change can double-apply
• An idempotency failure on a leave entry can mis-balance PTO

The pattern: every commit-shaped action carries an idempotency key that uniquely identifies the intended outcome.

import hashlib, json

def idempotency_key(action_type: str, payload: dict) -> str:
    """Stable hash of the intended action. Same inputs → same key."""
    canonical = json.dumps(payload, sort_keys=True)
    return f"{action_type}:{hashlib.sha256(canonical.encode()).hexdigest()[:16]}"

# Before committing, store the key. If we see it again, no-op.
def commit_with_idempotency(action_type, payload, store):
    key = idempotency_key(action_type, payload)
    if store.exists(key):
        return store.get(key)["result"]  # already done
    result = actually_commit(payload)
    store.put(key, {"result": result, "ts": time.time()})
    return result

Prompt caching — the cost lever you can't ignore

Anthropic's prompt caching gives ~90% discount on cached input tokens. The cache lasts ~5 minutes by default with longer-lived options. In HR, the cacheable surfaces are huge:

• The policy library (often 100K+ tokens)
• The Workday schema and field dictionary
• System prompts for each agent
• Country-specific leave entitlement reference data
• Role rubrics for promotion review

Architecture rule: stable content goes first in the message array; per-call variable content goes last. The cache breakpoint sits at the join.

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=4096,
    system=[
        {
            "type": "text",
            "text": POLICY_LIBRARY_FULL,  # 80K tokens, stable
            "cache_control": {"type": "ephemeral"},
        },
        {"type": "text", "text": TASK_SPECIFIC_INSTRUCTION},
    ],
    messages=[
        {"role": "user", "content": user_question}  # variable
    ],
)

Model selection — when each is right

In interview: "I default to Sonnet, drop to Haiku for high-volume triage, and reach for Opus when the task spans multiple jurisdictions or has genuinely ambiguous reasoning."

Streaming & latency

For chat-shaped UX (e.g. an HRKX agent over Slack) you'll want streaming. For batch workflows (e.g. nightly survey sweep) you don't. Two notes:

• Streaming complicates structured-output validation — buffer the full response, then validate.
• For HITL workflows, time-to-first-token matters less than time-to-correct-output. Don't over-optimize streaming.

Cost shape — what drives it

Two levers dominate cost in agent systems:

• Input tokens — usually 10-100x more than output. Cache the stable parts aggressively.
• Loop length — multi-turn tool use compounds. Cap turns. Audit logs should show turn counts per run; a sudden growth is a regression.

For a typical HR agent run: 50K cached input tokens + 5K variable input + 2K output × ~3 turns = your unit cost. Know roughly what one run costs and what a month at production volume costs. Engineering leaders ask.

Fluency check — answer these out loud

1. What's the difference between tool use and structured output? When would you use one but not the other?
2. Why does prompt caching matter for an HR Q&A agent specifically?
3. Give an idempotency key design for "process a leaver checklist for worker W-1234 effective 2026-06-30."
4. What's a retry policy for a "propose EIB upload" tool call vs. a "send Slack welcome message" tool call? Why different?
5. Which Claude model would you reach for to synthesize 200 free-text survey responses with sentiment + theme clustering? Why?

If those are crisp, move to 04-deep-dive-primary.

✓ Check yourself

Answer from memory before opening each one.

1. In the agent loop, the model proposes a tool call and your code runs it. Why does it matter that the orchestrator, not the model, controls the loop, in a workflow that ends in a Workday write?

   Show answer

   Because the orchestrator can refuse, log, cap turns and turn the final write into an approval request, and the model cannot. The model only emits a proposal. Every tool call becomes an audit entry written by your code on entry and exit, and the last step becomes a HITL request instead of a direct write. See LLM orchestration.

2. A tool called propose_job_change returns a preview and writes nothing. Where is the approval boundary enforced, and what two things make it hold?

   Show answer

   At the tool boundary. The description tells the model the tool does not commit, and the implementation really only drafts an EIB row and returns a preview. The prompt wording alone would not hold, so the second guarantee is that the code behind the tool has no write path. Server-side validation covers bad inputs on top. See Tools / function calling.

3. The model returns a proposed job change with an FTE of 1.4 and a reason code not in your list. What does the system do, and how does that differ from what it does for a 503 from the API?

   Show answer

   A schema failure on a payroll-adjacent action is a halt event: nothing reaches the approver, the failure is logged and surfaced to ops, with at most one corrective retry. A 503 is transient, so it gets exponential backoff with jitter for a few attempts. Logical and transient failures get different policies. See Structured output.

4. A network timeout hits after your code sent a Workday hire for a new employee, and the retry layer fires. What stops a duplicate worker record, and what should the key encode?

   Show answer

   An idempotency key stored before commit, so the second attempt finds it and returns the earlier result. The key should encode the business intent, such as the action type, worker and effective date, so the same hire retried five minutes or five hours later maps to the same key. A request-level ID would not catch a re-submitted intent. Still, a timeout after partial side effects is a case for failing closed and paging a human, not an automatic retry. See Idempotency.

5. An HR Q&A agent loads 80K tokens of policy text for every question. Where in the request do you put it, and what happens to cost if a team member reorders the prompt so the user question comes first?

   Show answer

   Put the stable policy library at the start, with a cache breakpoint after it, and the question and task instruction at the end. Caching works on a shared prefix, so putting variable content first changes the prefix on every call and you pay full price for the library each time instead of the discounted cached rate. See Prompt caching.

6. Your team needs to triage thousands of tickets a day, draft onboarding plans, and scope a data-subject access request spanning five countries. Assign a model tier to each and say what would change your mind.

   Show answer

   Use a small model for triage, the mid-tier model for onboarding drafts and tool use, and the largest for the multi-jurisdiction scoping. The deciding factors are volume, how ambiguous the reasoning is, and the cost of a wrong answer. Move a task up a tier when evals show the cheaper model failing on it, and down when the evals show no loss. See Model selection.

Takeaway

• The orchestrator owns the loop: the model proposes tool calls, your code executes them, logs each one, and turns the final write into an approval request.
• The approval boundary lives in the tool: a propose tool drafts and previews, has no write path, and says so in its description.
• Validate every structured output before a person or system sees it; for payroll-adjacent actions an invalid output is a halt event, not a silent retry.
• Retry transient failures with backoff and jitter, retry logical failures once with a corrective message, and never auto-retry a write with partial side effects.
• Idempotency keys should encode the business intent, so a repeated hire, comp change or leave entry is a no-op.
• Put stable system prompt and knowledge first and cache it; variable input goes last, and loop length is the other cost driver to cap.

The primitives are in place; next is how they compose into a full HR workflow with gates and audit. → Agentic Workflow Architecture for HR

Further reading

• Define tools — Anthropic docs. Tool definitions, the tool_use and tool_result loop, and how descriptions guide the model.
• Structured outputs — Anthropic docs. Getting schema-conformant JSON from the model, and what still needs your own validation.
• Prompt caching — Anthropic docs. Cache breakpoints, lifetimes and pricing for the stable prefix.
• API errors — Anthropic docs. Which status codes are transient and how to treat each.
• Writing effective tools for agents — Anthropic, 2025. Naming, descriptions and response design that make tools easy for a model to use.
• Models overview — Anthropic docs. The current model lineup with capabilities and pricing, for the tier choice.