Day-Of Tactics
The morning-of checklist, the structural moves that make design rounds easier, the traps to avoid, and the questions to ask them. Reread the morning of.
Night-before checklist
- Read chapters 01 (the role) and 16 (this one). Skim 04 (serving) and 05 (scheduling).
- Re-skim the 15-questions answer keys. Speak the answers aloud for the ones you flagged.
- Set up the room — water, notebook, pen, charger, second screen if you have one. Confirm video/mic.
- Have ready: a one-page "about me" bullet list with three concrete projects with numbers. Open it during the call if needed.
- Sleep. The marginal value of one more chapter at 1am is negative.
Morning of
- Reread this chapter. Spend 10 minutes.
- Caffeine on your usual cadence — interview day is not the day to try a new dose.
- Light food. A heavy meal in the hour before makes you slow.
- Walk ten minutes before the first round. Get blood moving.
- Tab close. Close Slack, email, anything that pings. Phone face-down.
- Bathroom break right before. Sounds dumb; matters.
- Five-minute warmup: read your "about me" out loud once; speak any one chapter-15 answer aloud.
The opening minutes
Most rounds start with "tell me about yourself" or some variant. You should have a 60-90 second answer rehearsed. Don't recite — structure.
- 10 seconds — identity. "I'm an infra engineer with N years on production systems, most recently focused on [X]."
- 30 seconds — strongest evidence. One project: scale, problem, what you owned, outcome with a number.
- 20 seconds — current depth. What you've been working on most recently, ideally in this role's vocabulary.
- 20 seconds — why this role. Specific to this team's mandate. Not "I love AI."
- 10 seconds — hand back. "What part would be most useful to dig into?"
The design-round structural move
Every design round in this loop follows roughly the same shape. Memorize the move.
- Clarify the workload. Online or batch? Interactive or async? Open-weights or proprietary? What's the latency target? What's the throughput target? What's the request profile (prompt length, output length)?
- Estimate compute. Order-of-magnitude: at target RPS and target throughput per GPU, how many GPUs? Mention Little's Law if it applies.
- Pick the serving stack. vLLM / Triton / TensorRT-LLM / KServe / Ray Serve — chosen with a reason rooted in step 1.
- Wire orchestration. K8s, node pool by accelerator, autoscaling signal, priority class, scheduling policy.
- Discuss failure modes. What goes wrong? How do you detect it? How do you recover? Be specific: OOM, NCCL, replica death, queue overflow.
- Cost. Reserved/spot mix; per-million-tokens estimate; idle headroom; "where does waste hide."
- Hand back. "Where would you like me to go deeper?"
This sequence is the senior frame. You can stop early if the interviewer pulls you sideways, but always come back to "and the failure modes / cost" before declaring done.
Coding round moves
- 60 seconds of setup. Restate the problem in your words. Ask one clarifying question. Name the data structure. Name the target complexity.
- Type the signature and data model first. The function header, the class fields, the inputs/outputs. Then bodies.
- Edge cases out loud. Empty, single-element, time going backward, capacity zero. List them before coding.
- Code in clean blocks. One responsibility per function. Names that read like the problem.
- Test as you go. A small assert at the bottom is worth two minutes.
- Discuss tradeoffs. "I'm using a heap because I need fast min and log insert. A sorted list would also work but the constant factor on append is worse."
- If stuck, narrate. "Let me think about whether the invariant holds when..." Silence is worse than narration.
Traps to avoid
- Don't over-promise on multi-tenant isolation. Be precise. "I can promise X with MIG; with MPS I can promise Y but not Z." Vague guarantees get torn apart.
- Don't claim a framework you haven't run. A single follow-up will expose it. "I've used X in benchmarks but not in production; my closest production reference is Y."
- Don't ignore queueing under burst load. Many candidates design happy-path-only systems. Bring up 429s, queue depth, load shedding before they ask.
- Don't conflate utilization metrics. Say "SM-active" or "DRAM-active" when you mean it. Generic "utilization" loses you the senior read.
- Don't say "GPU utilization is 100% so we're good" — for decode it can be 30% SM-active and 80% DRAM-active and that's saturation. Get the right number.
- Don't speed-read the prompt. Slow down. The fast take is rarely the senior take.
- Don't trash your previous stack. Senior engineers don't speak that way about systems they shipped.
- Don't go past the timebox without checking in. "I have a working solution; would you like me to optimize or move on?"
- Don't fake confidence on the cost math. "My rough estimate is $X; I'd want to validate against actual utilization curves" is fine.
- Don't forget the cost-perf-reliability triangle. Name which corner you're optimizing for and why.
Recovery patterns
You will say something wrong. Recover cleanly.
- Spot it yourself first. "Wait — I said X earlier; that's not right. The correct framing is Y." This is a positive signal, not negative.
- If the interviewer corrects you: thank them briefly, integrate, move on. "Right — thank you, that's the bit I had backwards." Don't dwell.
- If you don't know a term: say so. "I haven't run X. My closest reference is Y. Want me to reason about X from first principles, or would Y be more useful?"
- If you go silent for 20 seconds: narrate. "I'm thinking about whether the queue can deadlock if..." Silence is worse than a half-formed thought.
- If you go down a wrong path: pivot openly. "Let me reset — I think I was over-complicating this. The simpler version is..."
- If you're nervous: name it once, calmly. "Quick reset, I want to make sure I'm answering your actual question." Then continue. Mentioning nerves once humanizes; doing it repeatedly is a tell.
Questions to ask them
Pick three or four total across the loop. Adjust by round type.
For the hiring manager
- "What's the fleet shape today — node types, scale, mix of training vs inference — and what's the next hardware change you anticipate?"
- "What workloads are running in-house today versus going to external APIs, and what's the criteria for moving a workload from external to in-house?"
- "Where is the biggest unsolved problem in the team's first six months? Where would I have the most leverage if I joined?"
- "What does success look like for this hire at 30 / 90 / 180 days?"
For peers / IC interviewers
- "What does an on-call shift on this team look like? What fraction is driver/runtime vs scheduling vs serving vs capacity?"
- "Walk me through the last serious incident the team dealt with — what made it hard?"
- "What's the current GPU utilization picture — average SM-active, reserved vs spot, where are the biggest idle pockets?"
- "What tooling do you wish existed that doesn't yet?"
For senior leadership
- "How does this team's roadmap interact with the broader AI strategy at the company?"
- "What signals would tell you, six months from now, that this team is succeeding?"
Closing statement
End each round with a brief, specific signal of interest. Not "great chat, thanks." Try:
"This was useful — the [specific topic we discussed] is exactly the kind of problem I want to be working on, and the way you described the team's mandate is what I was hoping to hear. I'd be excited to keep the conversation going."
It's a sentence. It costs nothing. It often shows up in the interviewer's notes as "candidate seemed motivated."
After the loop
- Write down what was asked — within an hour, before you forget. Useful for follow-ups and useful for your next loop.
- Note anything you flubbed — that's your study list for next time.
- Send a short thank-you to the recruiter or hiring manager within 24 hours. One paragraph; mention one specific thing from the conversation.
- Don't relitigate in your head. You did what you did. If there's a second round, you'll have new chances.
You are interviewing them as much as they are interviewing you. The job has to be one you want to do, on a team where you'd grow, for a company whose direction you believe in. Posture matters; the calm of "this is a two-way conversation" produces better answers than the urgency of "I have to win this."