Choosing a Provider for Data Pipelines
Picking the right neocloud is one of the highest-leverage DE decisions in an AI organization. The choice shapes data movement, cost structure, orchestration, security model, and the long tail of operational decisions for the next year.
Decision axes
Seven dimensions matter most for a DE evaluating providers:
- Reliability and SLA. Whether jobs run end-to-end without intervention. Critical for production; tolerable for research.
- Network / cluster fabric. Whether you can run multi-node training. Required for serious training; irrelevant for single-node fine-tuning.
- Storage architecture. Whether the provider has fast, scalable storage adjacent to the GPUs. Matters enormously for training-data throughput.
- Pricing model. Per-second on-demand vs reserved vs spot. Each shapes how you design pipelines.
- Geography and data residency. Where the GPUs live relative to your data sources. Latency and compliance.
- Compliance posture. SOC 2, HIPAA, FedRAMP. Required for some regulated workloads.
- Integration surface. APIs, CLI, orchestrator support. How well it fits your stack.
Workload-to-provider map
Mapping workload types to provider classes (covered in the Neocloud topic guides):
| Workload | Best fit | Acceptable | Avoid |
|---|---|---|---|
| Multi-thousand-GPU training | CoreWeave, Crusoe, Nebius | Lambda for smaller | Marketplaces |
| Fine-tuning (small-to-mid models) | RunPod Secure, Lambda | Marketplace for cost-sensitive | Hyperscaler list prices |
| Production inference at scale | Together.AI, CoreWeave, hyperscalers | Lambda, RunPod Secure | Marketplaces |
| Sparse inference | Together.AI, RunPod Serverless, Hyperbolic | — | Dedicated for low-utilization |
| Research / experimentation | Vast.AI, RunPod Community, TensorDock | Lambda | Multi-year reservations |
| Indie / single-developer | Vast.AI, RunPod Community, Hyperbolic marketplace | — | Enterprise neoclouds |
| Regulated data workloads | Hyperscalers, CoreWeave with compliance contract | Lambda with attestation | Marketplaces, generally |
Data-specific considerations
From the DE's perspective specifically — beyond the general workload fit — these matter:
Storage adjacency
How fast can you get TBs of training data to the GPU? Providers vary:
- CoreWeave, Crusoe, Nebius offer fast parallel storage in-cluster.
- Lambda has object storage and persistent volumes with reasonable performance.
- Marketplaces typically only have local NVMe; bring-your-own-storage at the network layer.
If your training reads 50 TB from S3 every job, you're either paying for slow loading or paying for fast loading. Pick the provider accordingly.
Egress and cross-cloud
Moving training data into the GPU cluster from external storage (your S3, your customer's warehouse) incurs egress. Where the data starts and where the GPU lives shape this cost.
Per-second vs per-hour billing
Per-second billing (RunPod, modern neoclouds) means pipeline startup overhead is cheap; per-hour billing (some older providers) means you should batch work to amortize startup.
Spot / interruptible handling
If you use spot or interruptible instances, your pipeline has to handle preemption gracefully. Some providers give better preemption notice than others.
Multi-tenant isolation
Marketplace providers have weaker isolation than dedicated. If your data is sensitive, encrypt at rest and don't use marketplaces for the sensitive workloads.
Multi-provider as default
Most serious AI organizations operate across multiple providers. Reasons:
- Capacity constraints — no single provider has all the H200s you might need at the moment you need them.
- Negotiating leverage — single-provider lock-in gives the provider pricing power.
- Geographic distribution — different providers cover different regions.
- Cost optimization — different providers are cheapest for different SKUs at different times.
- Resilience — outages happen.
DE has to design for this. Abstracting over providers — through orchestrator-level adapters, through Kubernetes operators, through custom job submission services — is foundational work.
Contracting and commitments
Reserved-capacity contracts are usually negotiated by procurement / leadership, but DE input matters:
- What hardware mix you actually need over the term.
- What flexibility (region, scaling, GPU type changes) is required.
- What technical integration commitments the vendor is making (storage, networking, orchestration support).
- Exit and migration provisions if the relationship doesn't work.
The DE who builds the multi-provider abstraction can also help leadership understand exactly what they're committing to and what's reversible.
A working decision template
A reusable checklist when evaluating a new provider:
- What's the primary workload class this provider will serve?
- What's the cost gap to alternatives (on-demand list, reserved, after negotiation)?
- What's the storage architecture and how does it match our data layout?
- What's the network fabric and does it match our cluster sizes?
- What's the orchestrator integration story?
- What's the compliance posture and does it cover our workloads?
- What's the support model (response times, escalation paths)?
- What's the migration cost in if it doesn't work?
- How does it fit into the multi-provider posture we already have?
- What does the renewal / pricing-change story look like?
Takeaway
Provider choice is the foundation of every subsequent DE decision on neocloud infrastructure. The decision is rarely "best provider overall" — it's "best provider for this workload class given our broader posture." The next chapter looks at the most data-intensive part of the work: training-data pipelines.