Day-Of Tactics

Pre-loop checklist (90 minutes before)

1. Eat and hydrate. Hungry interviewees sound rattled.
2. Reread 01 — re-anchor on what the role actually is.
3. Reread 02 — re-anchor on posture.
4. Skim 14 — last vocabulary pass. The terms should slide off the tongue.
5. Pick 3 named past projects you'll cite. Have a one-sentence summary of each ready.
6. Rehearse your opening 90 seconds out loud, once.
7. Pick 2 questions to ask from below.
8. Check your environment. Camera, mic, charger, water, paper for sketches. Stable internet.
9. Stand up. Walk. Breathe. Cortisol drops in 10 minutes of movement.
10. Open one tab with the JD visible. Close everything else.

Structural moves — the shape of a good answer

Senior interviews reward structure. Random good content underperforms well-organized okay content. Two templates cover 80% of the loop:

The design-round template

When asked to design something — a gateway, an orchestrator, a tool router — walk these stages, out loud:

1. Clarify the SLO and workload. "What's the QPS, the latency target, the mix of short vs long context, the error budget?" If they don't have specifics, propose some and check.
2. Pick the concurrency model. "Async Rust + Tokio, work-stealing. Bounded channels everywhere for backpressure. tower middleware stack."
3. Sketch the data plane. Request flow from edge to backend and back. Where caches sit. Where state lives (Postgres for durable, Redis for ephemeral).
4. Handle backpressure. Bounded queues, semaphores per tenant, load shedding under saturation.
5. Discuss failure modes. What breaks first under load? Tail latency. KV cache pressure. A flaky backend. For each, what's the response — circuit breaker, hedging, route-away, fail fast.
6. Talk about observability. Traces, RED metrics, audit events. How would you debug an incident.
7. Mention guardrails. Kill switches, rate caps, idempotency keys, approval gates for high-tier actions.
8. Bench and validate. "I'd load-test against representative traffic, watch p99 climb with concurrency, look for the saturation knee."
9. Surface the open questions. "Things I'd want to figure out: prefix-cache hit rate distribution, the long-context fraction, whether sub-agents are common enough to need their own pool."

The coding-round template

1. Restate the problem in your own words. "Just to make sure — you want X, given Y, with constraint Z."
2. State two approaches and the tradeoff. "Naive is O(n²) and trivial; with a hashmap it's O(n). I'll go with the hashmap unless memory is constrained."
3. Sketch the data structures and signatures. Type signature first. fn rate_limit(&self, key: &str, n: u32) -> bool.
4. Code in small pieces. Talk while you type. Pause to check assumptions.
5. Run through edge cases. Empty input. Concurrent calls. Failure on the network. "What happens if the channel is closed?"
6. State the complexity. Time and space.
7. Identify the next step. "In production I'd add metrics, an idempotency key, a timeout. Want me to write any of those?"

Practice this template on the problems in 11 until the structure is automatic.

Traps to avoid

Recovery patterns when you're stuck

• You don't know the term. "I haven't used X. My closest reference is Y. Want me to reason about it from there, or would you prefer to give me the framing?"
• You realize your design is wrong mid-answer. Out loud: "Wait — that doesn't handle the partial-failure case. Let me revise. Here's the actual move..." Course-correcting visibly is a senior signal, not a junior one.
• You blanked on a detail. "I'd want to look up the exact tokio call here, but the shape is..." Then keep going. Don't dwell.
• You finished early. "I'm happy with that. Want me to walk through the failure modes I'd worry about, or take it deeper somewhere?"
• You're running out of time on coding. Pseudo-code the rest with comments. State the time complexity. Move on.
• The interviewer pushes back. "Tell me where you'd want this to go differently" — engage their reasoning, don't just defend.
• You misread a constraint. "Hold on, let me re-check — when you said X, did you mean Y or Z?" Better than building 20 minutes on the wrong base.

Questions to ask them

End every round with at least one. Pick situationally; here's a strong set:

1. "Where's the AI Infrastructure / Agent Systems boundary in practice today, and where is it the most fuzzy?" Shows you've thought about team topology.
2. "What's the current p99 first-token-latency SLO and the dominant tail contributor?" Shows latency-aware thinking.
3. "How do you currently handle backpressure between the orchestrator and the inference fleet during a spike?" Shows distributed-systems instinct.
4. "What's the audit/replay story for an autonomous agent action a regulator might later question?" Shows you take compliance seriously.
5. "Where's the biggest leverage for a new senior engineer in the first 90 days — gateway perf, orchestration reliability, or eval/observability?" Shows you're thinking about impact.
6. "What's your on-call rotation like for the gateway?" Shows you take ownership seriously.
7. "When something breaks at 2am, who owns it, and what does the post-mortem look like?" Shows you've been there.
8. "What's the biggest open architectural question the team is debating right now?" Shows you want to engage with the real work.

Closing statement

When the interviewer says "any final thoughts?" — have one. Specific, not generic.

Avoid generic enthusiasm ("this seems great!"). Specifics show you were paying attention.

After the round

• Within 30 minutes: write down what they asked, what you said, where you felt strong, where you stumbled. Memory degrades fast.
• Within 24 hours: send a short thank-you note to the recruiter (or to each interviewer if appropriate). One sentence on something specific from the conversation. Don't grovel.
• Between rounds: spot-fix one thing from your notes. If you blanked on prefix caching, reread 05. Don't try to fix everything.
• If you don't advance: ask for feedback. Most companies won't give it. Some will. It's worth asking.
• If you do advance: revise your notes, pick the 2-3 things you most want to improve before the next round, and drill those.

✓ Check yourself

Answer from memory before opening each one.

1. You are asked to design a tool router and start drawing boxes. Which step did you skip, and what do you ask for?

   Show answer

   Clarifying the SLO and workload. Ask for request rate, latency target, request mix and error budget, and propose numbers yourself if none are given. Without them you cannot choose between trade-offs such as queueing and shedding. See The design-round template.

2. Name three things that tend to break first in an inference gateway under load, and the response for each.

   Show answer

   Tail latency, KV cache pressure and a flaky backend. The responses are hedging or load shedding for the tail, routing away from a saturated backend, and a circuit breaker with fail-fast for the flaky one. Naming the failure and its mitigation as a pair is the habit to build. See The design-round template.

3. Your rate limiter compiles. What do you cover before saying it is done?

   Show answer

   Edge cases such as empty input, concurrent calls and a closed channel, then time and space complexity, then what production would add: metrics, an idempotency key and a timeout. Offer to write any of those if the interviewer wants. See The coding-round template.

4. Someone proposes reading a config file with a blocking call inside the request handler. What does that signal, and what is the better answer?

   Show answer

   It blocks a Tokio worker and suggests the engineer does not understand the runtime. Make I/O async, and reserve spawn_blocking for genuinely synchronous CPU work. See Traps to avoid.

5. Why is forgetting deadline propagation on the trap list, and what is the phrase that avoids the trap?

   Show answer

   A local timeout does not know how much time the caller has left, so downstream work continues after the request has been abandoned. Say that deadlines, not just timeouts, carry the request's remaining budget through every downstream call. See Traps to avoid.

6. Halfway through an answer you see that the design ignores partial failure. Do you finish and hope, or revise, and why?

   Show answer

   Revise out loud: name the gap and show the fix. Visible course-correction is a senior signal, while quietly finishing a design with a known hole costs more. If time is short, say what you would change and move on. See Recovery patterns.

Takeaway

• Do the pre-loop work: reread the role, the positioning and the vocabulary chapters, pick three projects, and rehearse the opening once.
• Design answers follow a fixed order: SLO and workload, concurrency model, data plane, backpressure, failure modes, observability, guardrails, validation and open questions.
• Coding answers restate the problem, state two approaches, sketch signatures, code in small pieces, cover edge cases and complexity, then name production next steps.
• Avoid the traps by replacing each with its stronger line: safe Rust first, p99 thinking, async I/O, bounded queues, idempotency keys and deadlines.
• When stuck, say what you know, ask whether to reason or to be given the framing, and correct yourself visibly.
• End each round with a question and a specific closing statement, then write notes within thirty minutes.

You have reached the end of this guide. To build the thing it describes, continue with the sibling guide, or return to the hub. → Build an inference gateway in Rust · All chapters