Picking the Right Provider for ML Work

DS-side decision differs from DE-side

The DE guide's provider chapter framed the decision around data movement, orchestration, and cost engineering. The DS perspective adds:

• Time-to-result matters as much as cost (iteration velocity).
• Hardware-specific performance matters (some workloads need specific SKUs).
• Tooling fit matters (does the provider's environment support your framework cleanly).
• Quality vs cost tradeoff differs by ML activity (research tolerates flakiness; production doesn't).

Research / experimentation

For exploratory ML work — quick training runs to test ideas, evaluation on a candidate model, debugging — the priority is cheap GPUs available immediately.

• First choice: Vast.AI for the lowest prices and broadest hardware selection.
• Second choice: RunPod Community Cloud for slightly more polish at modest premium.
• Third choice: Hyperbolic marketplace or TensorDock for alternative supply.
• Avoid: Reserved-capacity providers — overkill and slow procurement.

Interruptible / spot instances are ideal here. Lost work is small; cost savings are large.

Fine-tuning

For fine-tuning small-to-mid models (sub-70B, LoRA / QLoRA / full):

• Single-GPU or small multi-GPU: Marketplaces or RunPod Secure / Lambda.
• Larger fine-tunes (8-32 GPUs): Lambda, RunPod Secure, Together training clusters.
• Managed fine-tuning: Together.AI's fine-tuning service for if you want zero infrastructure work.

The choice depends on whether you want to manage the infrastructure (cheaper, more control) or pay a premium for managed (faster delivery).

Large-scale training

For training larger models from scratch or significant pre-train extensions (100+ GPUs in cluster):

• Top tier: CoreWeave, Crusoe, Nebius — multi-thousand-GPU cluster fabric, NVIDIA reference designs.
• Mid tier: Lambda's 1-Click Cluster for hundreds-of-GPU scale without multi-year contracts.
• Avoid: Marketplaces (no fabric for distributed training).

For multi-thousand-GPU training, reservations months in advance are typically required. This is procurement work as much as DS work.

Inference deployment

Production inference options:

• Managed per-token API: Together.AI, Hyperbolic inference, Fireworks. Best for spiky traffic and lower volumes.
• Dedicated endpoints: Together's dedicated, RunPod Serverless dedicated. For predictable medium-volume.
• Self-hosted on dedicated GPUs: Lambda, RunPod Secure, CoreWeave for high-volume / custom models / control needs.
• Spot / marketplace inference: For non-production or fault-tolerant batch inference workloads.

The break-even between managed and self-hosted depends on utilization. Below ~50% utilization, managed wins. Above ~70%, self-hosted wins. The middle is a judgment call.

Evaluation infrastructure

Running eval suites at scale (especially for LLMs with many benchmarks) is its own workload class:

• Often parallelizable across many smaller GPUs.
• Latency-tolerant (run overnight; check results in morning).
• Spot-friendly.

Marketplaces are well-suited. Some teams build dedicated eval clusters on Lambda or RunPod Secure for repeatability.

Takeaway

Provider choice from the DS / ML perspective varies activity by activity. There's no single "right" provider; matching activity to provider class is the skill. The next chapter goes deeper on training and fine-tuning workflows.