The DS Workflow on GPU Clouds

The shape of DS work

In a 2026 AI / ML organization running on neoclouds, DS / ML work has these characteristic activities:

• Research and experimentation. Trying ideas — new architectures, data mixes, training regimes — usually at small scale.
• Fine-tuning at scale. Customizing open-source or in-house models on customer-specific data.
• Large-scale training. The big training runs (full pre-train or significant fine-tunes).
• Evaluation. Building eval sets, running benchmarks, tracking metric movement.
• Production deployment. Serving models to customers via APIs.
• Monitoring. Watching production model behavior and feeding back insights.

Each activity has different infrastructure needs. Research uses cheap interruptible GPUs; large-scale training uses dedicated clusters; production inference uses either dedicated or per-token managed services.

The experiment-to-production cycle

The standard cycle:

1. Hypothesis: a problem worth solving or an idea worth testing.
2. Quick experimentation at small scale (single GPU or small cluster). Often on marketplaces (Vast, RunPod Community) for cost.
3. Iteration on the most promising approaches.
4. Scale up to a full training or fine-tune. Often on dedicated providers (CoreWeave, Crusoe, Lambda, Together training).
5. Evaluation against the eval set; iteration if needed.
6. Deployment to inference infrastructure (Together API, Hyperbolic inference, RunPod Serverless, or self-hosted on dedicated).
7. Production monitoring and feedback into the next cycle.

The cycle's velocity is shaped by how friction-free each step is. Friction at the small-experiment step (waiting for a GPU; provider issues) slows iteration. Friction at the production step (deployment complexity; cost) slows delivery.

How neoclouds change the workflow

Compared to a hyperscaler-default workflow:

• Cheaper experimentation. Marketplaces dramatically lower the cost of trying things. More ideas tested.
• More variability. Marketplace supply quality varies; planners have to allow for it.
• Better fabric available. Dedicated neoclouds have InfiniBand cluster fabric that's cheaper than hyperscaler equivalents.
• Different commercial structure. Reserved-capacity contracts for large training; per-token APIs for inference.
• Multi-provider posture. Most teams don't lock to one provider.
• Sharper cost focus. Compute is a bigger fraction of total spend than at most hyperscaler customers.

Roles and handoffs

In a mature org, roles partition like this:

• Research / applied DS: Hypotheses, experiments, evaluation, model improvement.
• ML engineers / infrastructure: Pipelines, training infrastructure, serving infrastructure, monitoring.
• Data engineering: Upstream and downstream data pipelines.
• Platform / FinOps: Cost reporting, provider relationships, reservation negotiation.

Smaller teams blur these. The DS who knows enough infrastructure to spin up a cluster and run a fine-tune unassisted is valuable; the DS who can't speak to provider economics is increasingly limited.

Takeaway

DS work on neoclouds is more cost-aware and more multi-provider-aware than hyperscaler-default DS. The next chapter formalizes the provider decision from the DS perspective.