Lab 08 — Serve It — BI Dashboard

The goal

Every lab so far has moved data toward a destination it never quite reached: a human making a decision. The generator produced events, ingestion landed them, dbt modeled them into a star schema, the rollup tamed the telemetry firehose, orchestration ran it on a schedule, tests guarded it, and Lab 07 defined the metrics. Now we close the loop. We're going to plug a business intelligence (BI) tool — Metabase — into the warehouse and build a dashboard for the imaginary people who run our GPU marketplace: the ops lead who cares about utilization, the finance lead who cares about GMV, the account manager who cares about top customers.

Here's the slice of the architecture this lab completes — the far-right "SERVE" column:

By the end you'll have a single dashboard with five charts — GMV over time, utilization by GPU model, fill rate, top customers by revenue, and telemetry health — all reading from the governed gold layer so the numbers match everywhere.

Connect Metabase to the warehouse

This is the one genuinely fiddly part of the lab, so let's be honest about it up front. Our warehouse is DuckDB — a fantastic embedded analytics engine, but an awkward fit for a server BI tool. Metabase has no built-in DuckDB driver; there's a community driver you can drop in, but it lags Metabase releases, needs a matching .jar version, and breaks when DuckDB has the file open from another process (DuckDB allows only one writer). On a laptop that means flaky "database is locked" errors right when you're demoing.

So you have two paths:

Step 1 — Publish gold to Postgres

We already have Postgres running (it's our source for rentals/customers, but a separate schema keeps serving data clean). The simplest robust approach: a tiny publish script that reads each gold table from DuckDB and writes it into a serving schema in Postgres. DuckDB can attach Postgres directly, so this is a handful of lines:

"""Publish gold marts + metrics from DuckDB into Postgres `serving` schema
so Metabase (which has no DuckDB driver) can read them natively."""
import duckdb, os

PG = os.environ["SERVING_PG_DSN"]  # e.g. postgresql://griddp:griddp@postgres:5432/griddp
DUCKDB_PATH = os.environ.get("DUCKDB_PATH", "warehouse.duckdb")

# Tables to serve: the gold star schema + the Lab 07 metrics view.
TABLES = [
    "dim_customer", "dim_gpu", "dim_date",
    "fct_rentals", "fct_telemetry_hourly",
    "metrics_daily",          # the governed semantic metrics from Lab 07
]

con = duckdb.connect(DUCKDB_PATH, read_only=True)
con.execute("INSTALL postgres; LOAD postgres;")
con.execute(f"ATTACH '{PG}' AS pg (TYPE postgres);")
con.execute("CREATE SCHEMA IF NOT EXISTS pg.serving;")

for t in TABLES:
    print(f"publishing {t} -> serving.{t}")
    con.execute(f"DROP TABLE IF EXISTS pg.serving.{t};")
    con.execute(f"CREATE TABLE pg.serving.{t} AS SELECT * FROM main_marts.{t};")

print("done — gold published to Postgres serving schema")
con.close()

Adjust main_marts to whatever schema your dbt gold models land in (check dbt_project.yml). Run it from the repo root:

export SERVING_PG_DSN="postgresql://griddp:griddp@localhost:5432/griddp"
export DUCKDB_PATH="warehouse.duckdb"
uv run python ingest/publish_to_serving.py

Step 2 — Point Metabase at the serving database

Open Metabase at http://localhost:3000. On first run it walks you through creating an admin account (use any email/password — it's local). Then add the database:

Build the dashboard

Metabase is a GUI, so there are no files to type here — but every chart is just a saved question, and behind every question is SQL. We'll give you the SQL for each so you understand exactly what the chart shows (and so you can paste it into Metabase's native SQL editor, which is the fastest path). For each: click + New → SQL query, pick the serving database, paste the SQL, run, choose the visualization, and Save. Then create a dashboard called "Marketplace Overview" and add each saved question to it.

1 — GMV over time (line chart)

GMV (gross marketplace value) is the headline number — total rental value transacted per day. Read it straight from the governed metrics table so it matches finance's number exactly.

SELECT day, gmv
FROM serving.metrics_daily
ORDER BY day;

Visualization: Line, x = day, y = gmv. Format the y-axis as currency.

2 — Utilization % by GPU model (bar chart)

The ops lead's number: of the hours each GPU model could have been rented, what fraction actually was. We derive it from the telemetry rollup, which knows when each GPU was busy.

SELECT g.gpu_model,
       ROUND(100.0 * SUM(t.busy_minutes)
             / NULLIF(SUM(t.total_minutes), 0), 1) AS utilization_pct
FROM serving.fct_telemetry_hourly t
JOIN serving.dim_gpu g ON g.gpu_key = t.gpu_key
GROUP BY g.gpu_model
ORDER BY utilization_pct DESC;

Visualization: Bar, x = gpu_model, y = utilization_pct. (If your rollup names columns differently, adjust — the shape is "busy ÷ available.")

3 — Fill rate (single number / trend)

Fill rate = of the rental requests the marketplace received, what share were filled by an available GPU. It's a marketplace-health metric: a low fill rate means demand you couldn't serve.

SELECT day, fill_rate
FROM serving.metrics_daily
ORDER BY day;

Visualization: Line for the trend, or a Number showing the latest day with a sparkline. Format fill_rate as a percentage.

4 — Top customers by revenue (table or bar)

The account manager's view: who are our biggest customers? Join the rentals fact to the customer dimension and sum revenue.

SELECT c.customer_name,
       c.tier,
       ROUND(SUM(r.revenue), 2) AS total_revenue,
       COUNT(*)                 AS rentals
FROM serving.fct_rentals r
JOIN serving.dim_customer c ON c.customer_key = r.customer_key
GROUP BY c.customer_name, c.tier
ORDER BY total_revenue DESC
LIMIT 10;

Visualization: Table (clean for a top-10), or a horizontal Bar with customer_name on the y-axis.

5 — Telemetry health (single number)

A platform-trust metric: are the GPUs actually reporting? If telemetry goes quiet, every utilization number above is suspect. We count how many GPUs reported in the most recent hour versus how many exist.

WITH latest AS (
  SELECT MAX(hour) AS h FROM serving.fct_telemetry_hourly
)
SELECT COUNT(DISTINCT t.gpu_key)            AS reporting_gpus,
       (SELECT COUNT(*) FROM serving.dim_gpu) AS total_gpus
FROM serving.fct_telemetry_hourly t, latest
WHERE t.hour = latest.h;

Visualization: Number (or two numbers side by side). When reporting_gpus drops below total_gpus, something upstream is dropping telemetry — exactly the kind of "is my data even arriving?" check Lab 09 will formalize.

Build on the governed layer

Look back at those five queries. Notice what they don't do: they never recompute a business definition in the BI tool. GMV and fill rate come straight from serving.metrics_daily — the metrics you defined once in Lab 07. Revenue and utilization read the gold facts and dimensions, not raw bronze. That discipline is the whole point of a governed dashboard.

The failure mode it prevents is the classic one: finance's slide says GMV was $4.1M, the ops dashboard says $3.8M, and a meeting gets eaten arguing about whose number is right. The cause is almost always that someone redefined "GMV" with a slightly different filter inside a BI question. When every chart reads the same marts and the same metrics table, that argument can't happen — the numbers agree because they come from one definition.

Run it & view the dashboard

Let's do a clean end-to-end pass, the way you would before showing it to someone. From the repo root, with the stack up:

# 1. rebuild gold + metrics so the warehouse is current
cd dbt && dbt build && cd ..

# 2. publish the fresh gold into the Postgres serving schema
uv run python ingest/publish_to_serving.py

# 3. (in Metabase) hit the refresh icon on the dashboard, or
#    Settings -> Databases -> serving -> "Sync database schema now"

Open http://localhost:3000 and go to your Marketplace Overview dashboard.

Commit Lab 08

Two things changed: the serving step (the publish script) and your dashboard's definitions. Metabase stores dashboards in its own database, but you should still commit the SQL and notes so the dashboard is reproducible from the repo — drop them in the dashboards/ folder the skeleton reserved.

# save each question's SQL + a short README under dashboards/
mkdir -p dashboards/marketplace-overview
# (paste each query into dashboards/marketplace-overview/*.sql,
#  and note the chart type + dashboard layout in a README.md)

git add ingest/publish_to_serving.py dashboards/
git commit -m "Lab 08: serve gold to Postgres + Marketplace Overview dashboard"

✓ Check yourself

• Can you explain why we publish gold to Postgres instead of pointing Metabase at DuckDB directly?
• For each of the five charts, can you name which gold table or metric it reads from?
• Why does reading GMV from metrics_daily (instead of recomputing it in a chart) prevent the "whose number is right?" argument?
• Where in the orchestration graph should the publish-to-serving step live, and why?

SELECT c.tier,
       ROUND(SUM(r.revenue), 2) AS total_revenue,
       COUNT(DISTINCT c.customer_key) AS customers
FROM serving.fct_rentals r
JOIN serving.dim_customer c ON c.customer_key = r.customer_key
GROUP BY c.tier
ORDER BY total_revenue DESC;

git add dashboards/marketplace-overview/revenue_by_tier.sql
git commit -m "Lab 08: add revenue-by-tier chart"

Next

The platform now serves answers — but a build that only works on your laptop isn't shippable. Next we add the safety net that makes it trustworthy: automated checks on every change and visibility into every run. → Lab 09 — CI/CD & Observability