Positioning From Scratch
Mindset before content. How to interview honestly for a GPU infrastructure role when your background is solid but doesn't perfectly map — classical infra without GPUs at scale, or HPC without modern LLM serving.
The premise
The strongest candidates for this role haven't necessarily run a 10,000-H100 fleet at a frontier lab. The pool is small and the field is young; most people preparing for this interview are coming in with a partial match. The team knows this. They're hiring for judgment, fundamentals, and learning velocity, not for a perfect resume.
If you have an interview scheduled, someone decided your conversation was worth their hour. They saw signal somewhere. Your job is to confirm the signal, not to bluff your way past gaps. Bluffing is the single fastest way to lose a senior infra round.
Two common archetypes
Most candidates I see fall into one of two buckets. Recognize yours; the right preparation is different.
| Archetype | Strengths | Gaps you'll need to close |
|---|---|---|
| Classical infra / SRE / platform — Linux, K8s, networking, observability, on-call, large-scale services, but limited GPU production time. | Operational maturity. Incident discipline. Fluency with K8s, Prometheus, deploy pipelines. Cost discipline. Comfort with "always-on" expectations. | GPU-specific failure modes. Inference serving frameworks (vLLM, Triton). NCCL/CUDA internals. Tensor parallel shapes. Quantization tradeoffs. |
| HPC / research compute / ML researcher-adjacent — Slurm, MPI, CUDA, distributed training, profilers, but limited production LLM serving and limited K8s. | GPU fluency. CUDA / NCCL debug experience. Profiling instincts. Understanding of parallelism strategies. Comfort with hardware. | Kubernetes orchestration. Service-level reliability. Multi-tenant scheduling. Cost attribution. SaaS-grade observability and incident response. |
If you're classical infra / SRE
You probably know more than you think. Translate your existing experience into GPU-shaped language:
- "Pod-level resource isolation" → MIG profiles, MPS, device plugin allocatable units.
- "Queue depth and saturation" — you already know these from web tier autoscaling. The signal is the same; the unit is tokens/second and SM-active%.
- "Driver/kernel upgrade with blast-radius control" — same playbook as a kernel CVE rollout, with extra care because the device plugin is sensitive.
- "Capacity planning under reserved-vs-spot mix" — same math you've done for EC2, with steeper unit cost and tighter reservation windows.
The vLLM paged-attention paper (skim), the Triton model repository docs (read), the NVIDIA k8s device plugin README (read), DCGM exporter metrics (skim). After that, do the chapter-11 coding problems on a timer.
Your honest framing in the interview:
"I've spent the last N years running production Kubernetes for general services — autoscaling, on-call, capacity planning, incident response. GPU clusters share the same operational shape; the surface area I'm closing now is the GPU-specific runtime layer — driver/CUDA management, NCCL debugging, inference serving stacks. I've done X this past quarter to close that. Happy to reason about a specific scenario with you."
If you're HPC / research compute
You bring the harder-to-fake side — actual GPU intuition. Translate your experience into the language of always-on production:
- "Slurm partition / QoS" → K8s namespace / priorityClass / ResourceQuota.
- "MPI all-reduce profile" → NCCL collective, same shape, different transport.
- "Job array submission" → batch inference queue with a worker pool.
- "Shared filesystem for checkpoints" → S3 / blob with locality cache, or Lustre/Weka if on-prem.
The gaps you should explicitly own:
- Kubernetes scheduling primitives (taints, tolerations, affinities, gang scheduling controllers).
- Service-level reliability — SLOs, error budgets, alerting, pages at 3am instead of "researcher emails me Monday."
- Multi-tenant cost attribution (chargeback per namespace, per team).
Your honest framing:
"My background is research compute — I've spent years debugging CUDA kernels, tuning NCCL ranks across fabric topologies, and profiling distributed training. The piece I'm building up on is the productionization layer: K8s scheduling, multi-tenant quota, SLOs and incident response for always-on inference. Here's where I am on each: ..."
Bridge language for both archetypes
Some phrasings work for either background. Borrow these.
| Situation | Bridge phrase |
|---|---|
| Asked about a tool you haven't used | "I haven't run X in production. The closest reference point I have is Y. Want me to reason about X from first principles, or would it be more useful to talk about Y?" |
| Asked about scale you haven't operated | "The largest fleet I've owned was N nodes. I'd expect the marginal complexity going to 10N to come from [scheduling fairness, fabric saturation, fault rate scaling]. Is that the direction you want me to explore?" |
| Asked a "have you ever" question with no | "No, but I'd approach it like this — [walk through the approach]. Where would you push on that?" |
| Asked to debug something exotic | "My instinct is to bisect — narrow what changed, narrow the failing surface, then reproduce in the smallest unit. Let me apply that here..." |
What not to do
- Don't claim a framework you haven't run. A single follow-up question will expose it. They've heard a lot of vLLM resumes; the people who've actually tuned it talk differently.
- Don't apologize for your background. One acknowledgement is enough. Then move on.
- Don't pretend to remember syntax under pressure. "I'd reach for the docs on the exact YAML, but the concept is..." is fine.
- Don't trash your previous stack. Even if you're done with it. Senior engineers don't speak that way about systems they shipped.
- Don't speed-read your way through a design question. Slow down. Name the constraints out loud. Ask one clarifying question. Then start.
The honest frame
"I'm a strong infra engineer who's been deliberately closing the GPU/inference gap. Here's the depth I have, here's where I'm still in flight, and here's how I'd approach problems where I'd be learning on the job."
Notice what this frame does:
- It claims senior identity without bluffing — "strong infra engineer."
- It signals intention and motion — "deliberately closing the gap."
- It invites the interviewer to map your skills to their needs — "where I am on each."
- It pre-empts the "but you haven't done X" trap by addressing it first.
A mini-script you can adapt for the opening
If the first question is the standard "tell me about yourself," have a 90-second answer ready. Structure:
- 10 seconds — who you are. "I'm an infra engineer with roughly N years on production systems."
- 30 seconds — the strongest evidence. One concrete project: scale, problem, what you owned, outcome with a number.
- 20 seconds — current depth. What you've been working on most recently, in the language of this role.
- 20 seconds — why this role. Specific to the company's mandate — privacy/latency/cost/differentiation for in-house compute, not "I love AI."
- 10 seconds — hand back. "What part would be most useful to dig into?"
Practice it out loud twice. Don't memorize it word-for-word. The point is structure under pressure, not recitation.