Day-Of Tactics

Night-before checklist

• Read chapters 01 (the role) and 16 (this one). Skim 04 (serving) and 05 (scheduling).
• Re-skim the 15-questions answer keys. Speak the answers aloud for the ones you flagged.
• Set up the room — water, notebook, pen, charger, second screen if you have one. Confirm video/mic.
• Have ready: a one-page "about me" bullet list with three concrete projects with numbers. Open it during the call if needed.
• Sleep. The marginal value of one more chapter at 1am is negative.

Morning of

• Reread this chapter. Spend 10 minutes.
• Caffeine on your usual cadence — interview day is not the day to try a new dose.
• Light food. A heavy meal in the hour before makes you slow.
• Walk ten minutes before the first round. Get blood moving.
• Tab close. Close Slack, email, anything that pings. Phone face-down.
• Bathroom break right before. Sounds dumb; matters.
• Five-minute warmup: read your "about me" out loud once; speak any one chapter-15 answer aloud.

The opening minutes

Most rounds start with "tell me about yourself" or some variant. You should have a 60-90 second answer rehearsed. Don't recite — structure.

1. 10 seconds — identity. "I'm an infra engineer with N years on production systems, most recently focused on [X]."
2. 30 seconds — strongest evidence. One project: scale, problem, what you owned, outcome with a number.
3. 20 seconds — current depth. What you've been working on most recently, ideally in this role's vocabulary.
4. 20 seconds — why this role. Specific to this team's mandate. Not "I love AI."
5. 10 seconds — hand back. "What part would be most useful to dig into?"

The design-round structural move

Every design round in this loop follows roughly the same shape. Memorize the move.

This sequence is the senior frame. You can stop early if the interviewer pulls you sideways, but always come back to "and the failure modes / cost" before declaring done.

Coding round moves

1. 60 seconds of setup. Restate the problem in your words. Ask one clarifying question. Name the data structure. Name the target complexity.
2. Type the signature and data model first. The function header, the class fields, the inputs/outputs. Then bodies.
3. Edge cases out loud. Empty, single-element, time going backward, capacity zero. List them before coding.
4. Code in clean blocks. One responsibility per function. Names that read like the problem.
5. Test as you go. A small assert at the bottom is worth two minutes.
6. Discuss tradeoffs. "I'm using a heap because I need fast min and log insert. A sorted list would also work but the constant factor on append is worse."
7. If stuck, narrate. "Let me think about whether the invariant holds when..." Silence is worse than narration.

Traps to avoid

• Don't over-promise on multi-tenant isolation. Be precise. "I can promise X with MIG; with MPS I can promise Y but not Z." Vague guarantees get torn apart.
• Don't claim a framework you haven't run. A single follow-up will expose it. "I've used X in benchmarks but not in production; my closest production reference is Y."
• Don't ignore queueing under burst load. Many candidates design happy-path-only systems. Bring up 429s, queue depth, load shedding before they ask.
• Don't conflate utilization metrics. Say "SM-active" or "DRAM-active" when you mean it. Generic "utilization" loses you the senior read.
• Don't say "GPU utilization is 100% so we're good" — for decode it can be 30% SM-active and 80% DRAM-active and that's saturation. Get the right number.
• Don't speed-read the prompt. Slow down. The fast take is rarely the senior take.
• Don't trash your previous stack. Senior engineers don't speak that way about systems they shipped.
• Don't go past the timebox without checking in. "I have a working solution; would you like me to optimize or move on?"
• Don't fake confidence on the cost math. "My rough estimate is $X; I'd want to validate against actual utilization curves" is fine.
• Don't forget the cost-perf-reliability triangle. Name which corner you're optimizing for and why.

Recovery patterns

You will say something wrong. Recover cleanly.

• Spot it yourself first. "Wait — I said X earlier; that's not right. The correct framing is Y." This is a positive signal, not negative.
• If the interviewer corrects you: thank them briefly, integrate, move on. "Right — thank you, that's the bit I had backwards." Don't dwell.
• If you don't know a term: say so. "I haven't run X. My closest reference is Y. Want me to reason about X from first principles, or would Y be more useful?"
• If you go silent for 20 seconds: narrate. "I'm thinking about whether the queue can deadlock if..." Silence is worse than a half-formed thought.
• If you go down a wrong path: pivot openly. "Let me reset — I think I was over-complicating this. The simpler version is..."
• If you're nervous: name it once, calmly. "Quick reset, I want to make sure I'm answering your actual question." Then continue. Mentioning nerves once humanizes; doing it repeatedly is a tell.

Questions to ask them

Pick three or four total across the loop. Adjust by round type.

For the hiring manager

1. "What's the fleet shape today — node types, scale, mix of training vs inference — and what's the next hardware change you anticipate?"
2. "What workloads are running in-house today versus going to external APIs, and what's the criteria for moving a workload from external to in-house?"
3. "Where is the biggest unsolved problem in the team's first six months? Where would I have the most leverage if I joined?"
4. "What does success look like for this hire at 30 / 90 / 180 days?"

For peers / IC interviewers

1. "What does an on-call shift on this team look like? What fraction is driver/runtime vs scheduling vs serving vs capacity?"
2. "Walk me through the last serious incident the team dealt with — what made it hard?"
3. "What's the current GPU utilization picture — average SM-active, reserved vs spot, where are the biggest idle pockets?"
4. "What tooling do you wish existed that doesn't yet?"

For senior leadership

1. "How does this team's roadmap interact with the broader AI strategy at the company?"
2. "What signals would tell you, six months from now, that this team is succeeding?"

Closing statement

End each round with a brief, specific signal of interest. Not "great chat, thanks." Try:

"This was useful — the [specific topic we discussed] is exactly the kind of problem I want to be working on, and the way you described the team's mandate is what I was hoping to hear. I'd be excited to keep the conversation going."

It's a sentence. It costs nothing. It often shows up in the interviewer's notes as "candidate seemed motivated."

After the loop

• Write down what was asked — within an hour, before you forget. Useful for follow-ups and useful for your next loop.
• Note anything you flubbed — that's your study list for next time.
• Send a short thank-you to the recruiter or hiring manager within 24 hours. One paragraph; mention one specific thing from the conversation.
• Don't relitigate in your head. You did what you did. If there's a second round, you'll have new chances.

✓ Check yourself

Answer from memory before opening each one.

1. You are asked to design serving for an internal assistant at 50 requests per second, with 600-token prompts, 300-token outputs and a 1-second p95 TTFT. What do your first two moves produce?

   Show answer

   First clarify the workload: interactive or async, open-weights or proprietary, and the latency and throughput targets. Then estimate compute, using the request profile and a rough per-GPU rate, and Little's Law for concurrency. Those two numbers justify the stack choice that follows. See the design-round move.

2. The interviewer pulls you sideways into a detail halfway through a design. Why do you still return to failure modes and cost before declaring done?

   Show answer

   Happy-path-only designs are the common failure, and failure modes and cost are where seniority shows. Returning to them completes the frame, and you can stop early elsewhere. See the design-round move.

3. You say a decode service is fine because utilisation is 100 percent. What is wrong with the sentence, and what do you say instead?

   Show answer

   Generic utilisation is ambiguous, and the nvidia-smi figure only shows that a kernel ran. Say SM-active or DRAM-active. For decode, 30 percent SM-active with 80 percent DRAM-active is saturation, so name the bottleneck resource. See traps to avoid.

4. Twenty minutes into a 45-minute coding question you have a working brute-force solution. What do you say?

   Show answer

   Say what works and its complexity, then check in: ask whether to optimise or move on. Going past the timebox without checking in is a listed trap. If you optimise, name the structure and target complexity first. See traps to avoid and coding round moves.

5. Five minutes ago you called decode compute-bound. You now realise that was wrong. What do you do?

   Show answer

   Say so yourself, plainly, and give the correct framing, then continue. Self-correction reads as a positive signal, while silent rewriting or defending the error does not. If the interviewer corrects you, thank them briefly and move on. See recovery patterns.

6. You have three conversations in a loop: hiring manager, a peer and a director. Which questions go to which, and why?

   Show answer

   Give the manager questions on fleet shape, workload criteria and success at 30, 90 and 180 days, the peer questions on on-call and the last hard incident, and the director questions on roadmap and success signals. Match each question to what that person can actually answer, and choose three or four in total. See questions to ask them.

Takeaway

• Prepare the night before, and keep the morning light: reread this chapter, eat lightly, close the notifications.
• A design round follows one move: clarify the workload, estimate compute, pick the stack, wire orchestration, discuss failure modes, cost, then hand back.
• A coding round starts with 60 seconds of setup, the signature, edge cases out loud, small tests and trade-offs.
• Name the precise metric, such as SM-active or DRAM-active, and bring up queues and load shedding before you are asked.
• Say what you have run and what you have not, and keep isolation promises to what each mechanism guarantees.
• A self-caught error is a positive signal. Narrate when stuck, and pivot openly when a path is wrong.
• Ask questions matched to the person, end each round with a specific signal of interest, and write down what was asked within the hour.

That is the full guide. For the memory system underneath serving, the sibling guide builds paged attention from first principles. → PagedAttention from First Principles