Start Here
AI engineer interviews in 2026 have converged on a small set of questions. This guide works sixteen of them, one per chapter, on a single running example so the answers reinforce each other. Start here for how the loop is scored, who Harbourline is, and how to get the most out of each chapter.
What the loop tests
Whatever the title says (AI engineer, applied AI engineer, LLM platform engineer, forward-deployed engineer), the loop is built around one worry: can this person put a probabilistic model into a system that other people depend on, and know when it is working? Every round is a different angle on that worry.
| Round | What it looks like | What it is really checking | Chapters |
|---|---|---|---|
| Technical screen | Retrieval and prompting questions, 45 minutes, often with a "why did this RAG answer go wrong" scenario | Do you reason from mechanism, or from vocabulary? | 01, 02, 03 |
| System design | "Design a customer-service agent" / "a batching system" / "a voice app", on a whiteboard | Do you clarify requirements, name trade-offs, and say where it breaks? | 05, 06, 09, 10, 11, 12 |
| Live debugging | An agent that loops, a contract that flakes, traces on screen | Can you bisect a stochastic system with evidence? | 13, 14 |
| Evals and judgement | "How do you know the change is safe to ship?" | Do you measure before you believe? | 07, 08, 15 |
| Behavioural | "Walk me through a system you shipped and its worst failure" | Did you own a real thing end to end? | 16 |
Coding rounds still exist and are usually ordinary: data structures in Python, sometimes an async retry loop or a small pipeline. They are not covered here; the data-engineering and compliance guides on this site have drill sets if you need them.
How answers are scored
Interviewers in these loops are rarely working from a key. They are listening for five things, and a strong answer shows all five in the first two minutes:
- Scope. You ask what the system is for, who is harmed when it is wrong, and what the volumes are, before you design. "What happens if this answer is wrong?" is the single highest-value question you can ask in any round.
- Mechanism. You explain why something fails (the embedding never saw the policy number as a token; the model cannot see a passage ranked eleventh when k is ten) rather than naming a fix from a list.
- Trade-off. Every recommendation comes with its price and the case where it flips. A reranker buys recall and costs 150 ms. Say both.
- Measurement. You name the metric, the set it is measured on, and the number that would change your mind.
- Failure instinct. You reach for "what breaks first" before "what is possible", and you put the human in the loop where the cost of error is highest.
The chapters give you a structured answer for each question. Do not deliver it as a monologue. State the structure in one sentence ("I'd split this into retrieval failures and generation failures and bisect"), then let the interviewer choose where to go deep. The "Interview probes" block in each chapter is the list of places they usually choose.
The running example: Harbourline Insurance
Every chapter uses the same fictional company, so that the design in chapter 06 reuses the retrieval you debugged in chapter 01 and the evals from chapter 07 decide whether chapter 15's fix shipped. Harbourline is a mid-size property-and-casualty insurer with about two million policies across the US and UK. Its AI programme has six surfaces:
| Surface | What it does | Volume | Cost of a wrong answer |
|---|---|---|---|
| Help-centre assistant | RAG over policy documents and internal procedures | Part of ~40k chat sessions/day | A wrong deductible quoted to a customer: a complaint, a refund |
| Claims-intake agent | Reads a first notice of loss, asks for what is missing, checks coverage, routes to an adjuster | ~6k new claims/day | A wrong coverage decision: a regulatory issue |
| Customer-service agent | Web chat with tools for policy lookup, payment status, address changes | ~40k sessions/day | A wrong write (address, payment): real harm, hard to undo |
| Voice line | The FNOL phone line, moving from IVR to voice AI | Thousands of calls/day | A dropped or misheard claim detail |
| Document uploader | Claim photos, dashcam video up to 2 GB, policies up to 300 pages | Per claim | Lost evidence |
| Nightly batch | Classifies and summarises the day's claims for adjusters | ~6k claims/night | Cheap per item, expensive in aggregate |
The numbers are there so you can do arithmetic out loud. When a chapter says "at 40k sessions a day, caching the system prompt saves roughly X", that is the kind of sentence that separates a candidate who has run a system from one who has read about it. Use your own numbers in the interview; the point is to have some.
The sixteen questions, in order
The questions are the ones that appear in nearly every published account of 2026 AI engineer loops. The chapters are grouped the way interviewers group them. Each chapter stands alone, but the later ones assume the vocabulary of the earlier ones.
Section B · RAG and retrieval
| Chapter | The question |
|---|---|
| 01 | A RAG answer is wrong even though the correct document exists. How do you debug it? |
| 02 | How would you choose chunk size and overlap? |
| 03 | When would you use hybrid search and reranking? |
Section C · Agents
| Chapter | The question |
|---|---|
| 04 | When would you use an agent instead of a deterministic workflow? |
| 05 | Design an insurance-claims agent that uses RAG while controlling token cost. |
| 06 | Design a customer-service AI agent. |
Section D · Evals
| Chapter | The question |
|---|---|
| 07 | How would you build an eval set and decide whether a change is safe to ship? |
| 08 | How would you evaluate an agent beyond checking its final answer? |
Section E · LLM system design
| Chapter | The question |
|---|---|
| 09 | Design an end-to-end batching system for LLM queries. |
| 10 | How would you reduce LLM latency and cost at scale? |
| 11 | Design a file uploader for large multimodal files that feeds an AI chat application. |
| 12 | Design a real-time voice AI application. |
Section F · Production and debugging
| Chapter | The question |
|---|---|
| 13 | How would you integrate an LLM behind a reliable application contract? |
| 14 | An agent loops forever on some inputs. Find the problem and fix it. |
| 15 | A model gives confident but factually wrong answers in a high-risk workflow. What would you change? |
| 16 | Walk through an AI system you shipped, its biggest production failure, and how you measured the fix. |
Two of these go deeper elsewhere on the site. For retrieval mechanics beyond what an interview needs, see RAG Applied in the compliance guide; for inference-side batching, the PagedAttention deep dive and the inference gateway build.
How to work a chapter
Each chapter has the same shape, and the shape is the study method:
- The question and what it scores. Read this, close the page, and give your current answer aloud for two minutes. Notice where you run out.
- The mechanism. This is the part to understand rather than memorise. If you can explain why a failure happens, you can answer the variant you were not prepared for.
- The worked answer on Harbourline. Read it for structure, not words. Your own answer should use your own system.
- Interview probes. Turn on drill mode, answer each probe aloud, then open the model answer. These are the follow-ups interviewers actually ask.
- Check yourself. Scenario questions with folded answers. Answer before opening. If you miss one, the answer links back to the section.
- Takeaway and Further reading. The claims to carry into the room, and the primary sources if you want the depth.
Reading an answer and being able to give one under questioning are different skills. The probes and the Check yourself blocks only work if you speak the answer before you look. A silent "yes, I know that" is the most common way to arrive at the interview unprepared.
Study schedule
| Time you have | Plan |
|---|---|
| 7+ days | One section a day in order (B, C, D, E split over two days, F), then a final day redoing every Check yourself aloud. About 90 minutes a day. |
| 2–3 days | 01, 03, 04, 05, 07, 08, 10, 13, 15, 16. Read only the Takeaway of the rest. |
| Under 24 hours | 01, 04, 07, 16, in that order. They cover retrieval, agents, evals and the behavioural spine. Then the Takeaway of 10, 13 and 15. |
If the role is specialised, weight accordingly: an inference-platform role should spend its extra time on 09 and 10; a product-facing applied AI role on 05, 06, 12 and 15.
Takeaway
- Every round in an AI engineer loop is a different angle on one question: can you put a probabilistic model into a system people depend on, and know when it is working?
- Strong answers show scope, mechanism, trade-off, measurement and failure instinct within the first two minutes, then let the interviewer pick the depth.
- "What happens if this answer is wrong?" is the first question to ask in any design or debugging round.
- One running example with real volumes lets you do arithmetic aloud, which is the clearest signal that you have operated a system rather than read about one.
- The chapters are a method: answer first, read second, drill the probes aloud, then check yourself.
Start with the question that opens most technical screens: a RAG answer is wrong even though the right document is in the index. → The Wrong Answer When the Right Document Exists