Error Handling for AI Systems

Why AI error handling is different

Traditional services fail clearly: 500 errors, exceptions, timeouts. You handle them with retries, fallbacks, alerts.

AI systems also fail quietly:

• The model returns a confident-sounding but factually wrong answer.
• The model partially follows instructions.
• The model invents a citation.
• The model picks a tool that's syntactically right but semantically wrong for the task.
• The model produces malformed JSON 1% of the time.
• A tool result is corrupted; the model carries on as if it's fine.
• A prompt-injected document silently steers the agent.

Failure mode taxonomy — speak this fluently

When asked "what failure modes do you design for?" walk through these seven categories:

The retry pattern — done right

Naive retries make AI failures worse (you keep paying for the same hallucination).

def llm_call_with_retry(prompt, schema, max_attempts=3):
    last_error = None
    for attempt in range(max_attempts):
        try:
            response = llm.complete(prompt)
            validated = schema.parse(response.text)
            return validated
        except ValidationError as e:
            last_error = e
            # repair-prompt: include the bad output and the validation error
            prompt = build_repair_prompt(prompt, response.text, str(e))
            continue
        except RateLimitError:
            sleep(exponential_backoff(attempt))
            continue
        except APIError as e:
            if e.is_transient:
                sleep(exponential_backoff(attempt))
                continue
            raise
    raise MaxRetriesExceeded(last_error)

Key principles:

• Distinguish transient vs permanent. Don't retry on 4xx auth errors.
• Repair, don't blind retry. When a schema fails, show the model what was wrong.
• Cap attempts. 3 is normal. After that, escalate.
• Cap cost. Total cost budget per task — abort if exceeded.
• Different error → different strategy. Rate limit = backoff. Schema error = repair. Tool error = let model decide.

Idempotency for side-effecting tools

The hardest class of AI errors: the agent calls a side-effecting tool, gets a timeout, retries, and now there are two records.

For any tool that mutates state:

• Generate idempotency key client-side (UUID), pass to the tool.
• Tool checks: have I seen this key? If yes, return the prior result.
• Persist (idempotency_key → result) mapping with a TTL.

Human-in-the-loop as error handling

The big one for compliance. Build a system where:

• Low-confidence outputs route to human review.
• High-stakes decisions always require human confirmation, even on high-confidence outputs.
• Edge cases (novel patterns, ambiguous inputs, conflicting evidence) detected and escalated.
• Humans can correct, and corrections feed back into evals.

The agent should have an explicit escalate_to_human tool. It's not a failure to use it — it's the right behavior when the agent isn't sure.

Confidence and abstention

Models say things confidently regardless of whether they know. To make confidence meaningful:

• Self-rated confidence: ask the model to score its own confidence (calibrated against eval data — most models are overconfident; you map raw scores to actual probabilities).
• Multi-sample agreement: run the same query 3-5 times; if outputs disagree, treat as low confidence. (Self-consistency.)
• Grounding ratio: % of claims with retrieval support. Lower = riskier.
• Refusal as data: when the model declines or says "I don't have enough information" — that's useful. Surface it, don't suppress it.
• Abstention loop: agent has an "I can't answer this" path. Common in QA systems.

Validation layers — defense in depth

Production AI systems usually have 3-4 validation layers:

Don't skip layers. Each catches a different failure class.

Logging errors for AI specifically

In addition to traditional logs, capture:

• Full input prompt (or hash + reference) at the time of the call
• Model version and parameters
• Raw output (not just parsed)
• Validation errors if any (with the malformed output)
• Tool calls (name, args, result, error)
• Eval / grader scores when available
• User feedback (thumbs, edits, escalations)

This is your debugging substrate AND your audit trail. Without raw inputs/outputs, you can't reproduce or explain what happened.

Observability — what to monitor

Set alerts on rate-of-change, not just absolute thresholds. Sudden drop in tool success rate = investigate.

Failing safe in compliance contexts

• Tool error → don't proceed with partial info; escalate.
• Low confidence → escalate.
• Validation failure → escalate, log, do not auto-recover.
• Cost / step budget exceeded → halt, escalate.
• Drift / eval regression detected → revert to prior model/prompt, page on-call.
• Unfamiliar input pattern → escalate.

When in doubt: pause. A regulator never penalized a firm for asking a human to double-check.

Chaos engineering for AI

Worth mentioning if you want to sound senior:

• Inject malformed tool results to verify the agent recovers.
• Force timeouts on a tool — does the harness fall back?
• Replay traces with corrupted prompts — does validation catch it?
• Inject prompt-injection payloads in test docs — does the guardrail catch it?
• Model fallback drills — primary model down, do you switch?

Talking-point: "How do you handle errors in agentic systems?"

✓ Check yourself

Answer from memory before opening each one.

1. Malformed JSON, a rate-limit response and a tool that raised an error each get a different strategy. Name each, and say why a blind retry is wrong for the first.

   Show answer

   Malformed JSON gets a repair prompt that shows the model its bad output and the validation error. A rate limit gets exponential backoff. A tool error is returned as a structured result so the model can react. A blind retry of the first case pays again for the same mistake. See the retry pattern.

2. An agent calls a tool that sends a customer notice, times out, retries, and two notices go out. What mechanism prevents this, and where is the key made and stored?

   Show answer

   An idempotency key. The caller generates it, for example a UUID, and passes it to the tool, which checks whether it has seen the key. If so it returns the earlier result. The key-to-result mapping is stored with an expiry. See idempotency.

3. List the conditions under which an EDD drafting agent should stop and escalate rather than carry on.

   Show answer

   A tool error that leaves partial information, low confidence, a validation failure with no auto-recovery, an exceeded step or cost budget, a detected drift or eval regression, and an unfamiliar input pattern. The default failure in compliance is more human review. See failing safe.

4. An answer cites a regulation that does not exist, yet nothing raised an exception. Why can try/except not catch this, and what does?

   Show answer

   The output is well-formed, so no error occurs. Quality failures are caught by layers: evals offline, then runtime semantic validation, a grounding check that each claim is backed by retrieved text, and output guardrails. See validation layers.

5. The model returns a self-rated confidence of 0.9. Do you use it to auto-approve?

   Show answer

   Not as given. Self-rated confidence is usually overconfident, so calibrate it against eval data, and combine it with agreement across samples and the share of claims that are grounded. For high-stakes decisions a human confirms whatever the score. See confidence and abstention.

6. Average tool calls per task jumps from four to eleven overnight, while the error rate holds steady. What does it suggest, and where do you look?

   Show answer

   Likely stuck loops, bloated context or a prompt, model or tool change that makes the agent take more steps. Read recent traces, compare the per-class error rates and check what changed. Alert on rate of change, not only fixed thresholds. See what to monitor.

Takeaway

• AI systems fail quietly, so try/except is not enough; you need layered validation and evals.
• Each failure class gets its own strategy: backoff for infrastructure, repair prompts for format, structured errors for tools, evals and checks for reasoning, guardrails for safety, limits for loops.
• Retry with attempt and cost caps, and show the model what was wrong instead of retrying blind.
• Every side-effecting tool takes an idempotency key, because a retried filing is a second filing.
• Escalating to a human is correct behavior, and the agent needs an explicit tool for it.
• Compliance fails safe: when unsure, the system produces more human review, never less.
• Log raw inputs, outputs, versions and validation errors with personal data redacted, because the same record serves debugging and audit.

The log is both a debugging tool and the heart of the audit story, which the next chapter treats in depth. → Audit Trails for AI Systems

Further reading

• Exponential Backoff And Jitter — AWS Architecture Blog, 2015. Why retries need randomized delay, with measurements.
• Circuit Breaker — Martin Fowler, 2014. The pattern for stopping calls to a failing dependency.
• Idempotent requests — Stripe API reference, accessed 2026. A well-known production design for idempotency keys on write operations.
• Tools — Model Context Protocol, 2025-06-18 revision. How tool errors are returned to the model.
• LLM01:2025 Prompt Injection — OWASP Gen AI Security Project, 2025. The adversarial failure class and its mitigations.