Marketplace / Usage-Data Domain
Domain fluency makes your SQL and modeling answers concrete. This chapter sketches what the data looks like at a usage-based, two-sided compute marketplace (think GPU rental) — kept general. Speak this language and you sound like you've already done the job.
Generic answers ("I'd build a star schema") are fine; domain-grounded answers ("I'd model usage at the instance-hour grain because that's how we meter and bill, and reconcile metered seconds against the billing system") signal you understand the business. That's a big edge as the person who'll own all their data.
A two-sided usage marketplace
The defining traits, and the data implications of each:
- Two sides: supply (hosts/providers offering GPUs) and demand (renters running workloads). You must measure and balance both — liquidity is the business.
- Usage-based billing: customers pay for what they consume (GPU-seconds, storage-GB-hours, egress). Revenue is metered, not a flat subscription — so metering accuracy is revenue accuracy.
- Perishable inventory: an idle GPU-hour is lost forever (like an empty airline seat). Utilization is the central efficiency metric.
- Heterogeneous supply: many GPU types, regions, reliability tiers, spot vs on-demand. Lots of dimensions to model.
- Heavy-tailed customers: a few large customers dominate volume. Medians and segmentation matter more than averages.
The data sources you'd stitch together
| Source | Carries | Gotchas |
|---|---|---|
| App / transactional DB | customers, instances, orders, plans, invoices | Normalized, rows update — needs CDC or updated_at incremental |
| Telemetry / metering pipeline | GPU-seconds, utilization, heartbeats per instance | High volume, at-least-once → dedup; late/out-of-order |
| Product / clickstream events | signup, launch, console actions | Schema drift, bot traffic, sessionization needed |
| Billing / payments system | charges, refunds, credits, payment status | Source of truth for revenue; must reconcile to metering |
| Infra / cloud cost | host costs, power, depreciation, bandwidth | Needed for margin; different grain & cadence |
Metering & usage — the heart of it
Metering converts raw telemetry into billable, reportable usage. It's where most data-correctness risk lives, because errors here are revenue errors.
- Grain: typically instance-second or instance-minute rolled to instance-hour. Decide the canonical grain and stick to it (chapter 06).
- Gaps & heartbeats: if an instance reports every 30s and a beat is missed, do you bill the gap? Define the rule (e.g. bill if next heartbeat within tolerance). This is a real judgement call with revenue impact.
- Duplicates: at-least-once telemetry means dedup in staging (chapter 04/05) or you over-bill.
- Partial hours & rounding: rounding rules materially move revenue at scale; document them and make them testable.
- Clock skew & late data: aggregate by event time with a lookback window so a late batch lands in the right hour.
- Multi-component cost: compute time, storage (often charged continuously, even while an instance is idle/stopped), and bandwidth each meter on their own grain. Total cost is the sum across components, not a single $/hr.
Utilization & capacity
The efficiency metric for perishable inventory: rented GPU-hours / available GPU-hours. You built this query in problem 11.
- Define "available" carefully: include or exclude machines in maintenance, offline hosts, or reserved-but-idle? The denominator choice changes the story — pick one and document it.
- Utilization is non-additive: store rented and available hours as additive facts and divide in the metric layer (chapter 06b) — never average pre-computed daily utilization.
- Segment it: utilization by GPU type, region, and host tier reveals where supply is mispriced or stranded.
- Capacity planning: forecasting demand vs supply to decide what hardware/regions to add. The data foundation is clean historical utilization by segment.
Billing & reconciliation
The highest-trust data in the company — finance and customers both depend on it. Two systems must agree: what you metered and what you charged.
- Reconciliation is a recurring DE task — metered usage priced out should match invoiced amounts within tolerance (problem 12). Gaps in either direction are leakage (lost revenue) or over-charge (customer trust + refunds).
- Credits, refunds, discounts, free tiers complicate "revenue." Model gross vs net, and recognize revenue on the right grain.
- GMV vs net revenue: on a marketplace, distinguish total customer spend flowing through the platform from the platform's own take/commission. Leadership cares about both; confusing them is a classic error.
- Disputes need point-in-time truth — "what rate was this customer on when this ran?" → SCD2 on pricing/plan plus an as-of join (chapters 06/02b).
"The first metric I'd make bulletproof is revenue, with an automated reconciliation between metering and billing — because it's the number leadership and finance check most, and the most damaging to get wrong."
Supply, demand & pricing
- Liquidity metrics: fill rate (demand met by available supply), time-to-provision, supply/demand ratio by GPU type and region. These tell you where the marketplace is tight or slack.
- Both-sides retention: churn of renters and of hosts. Losing supply is as damaging as losing demand.
- Price discovery: if interruptible/spot capacity is auctioned, the winning (clearing) price per GPU type/region/hour is a real price signal a fixed-price cloud doesn't have (03b #12).
- Price elasticity: how demand responds to price by segment — the analytical backbone of dynamic/spot pricing.
- Stranded supply: capacity that exists but isn't rentable (wrong region, wrong price, poor reliability score). Surfacing it is direct margin.
Unit economics
Leadership lives here; being able to model it makes you a partner, not just a pipeline-builder.
- Gross margin per GPU-hour: revenue per hour minus cost (host payout, power, depreciation, bandwidth). Needs the infra-cost source joined to usage — different grains, a real modeling exercise.
- Contribution margin by segment: by GPU type / region / customer tier — where you actually make money.
- Customer LTV & payback: usage-based LTV is noisy (heavy tails) — segment and use medians.
- Cost-to-serve: support, egress, failed-instance refunds attributed back to customers/segments.
Metrics glossary (speak the language)
| Term | Meaning |
|---|---|
| GPU-hour / GPU-second | The atomic unit of metered consumption |
| Utilization | Rented capacity ÷ available capacity |
| Fill rate | Share of demand satisfied by available supply |
| Time-to-provision | Latency from request to a running instance |
| Clearing price | The winning bid price on auctioned/interruptible capacity |
| Gross margin / GPU-hour | Revenue minus direct cost per hour |
| GMV vs net revenue | Total spend through the platform vs the platform's commission |
| Spot vs on-demand | Interruptible discounted capacity vs guaranteed |
| Churn (both sides) | Renters leaving; hosts withdrawing supply |
| Stranded capacity | Available but unrentable supply |
Now turn all of this into a plan you can present: 08 — Stack Decisions & 30/60/90 →