Course 6 · Capstone

Capstone Labs — Build Mini-GridDP

Everything comes together. Across ten labs you'll build mini-GridDP — a complete, running data platform for a GPU-rental marketplace, on your laptop: a data generator, ingestion, a modeled warehouse, a telemetry rollup, orchestration, tests, a semantic layer, a dashboard, CI/CD, and documentation. Each lab maps to a chapter of the senior Systems Design reference, so you build the very system that guide designs — at laptop scale.

This is your portfolio

By the last lab, your mini-griddp repo is a polished, end-to-end project you can show an employer: "I built and operated a data platform." It's the single most valuable artifact this curriculum produces. Work through the labs in order — each builds on the last.

Prerequisites

You need everything from Courses 2–5: the local stack (Course 2), the craft skills (Course 4), and the tooling/workflow (Course 5). If you've done those, you're ready. If you skipped ahead, do them first — this course assembles their pieces and won't make sense otherwise.

Labs

00 Start Here The mini-GridDP architecture, the repo layout, how the labs fit together, and how each maps to the Systems Design reference. Read first. Lab 01 Project Setup & the Data Generator Scaffold the repo and Compose stack, then write a Python generator that simulates the marketplace — customers, GPUs, rentals, events, and telemetry. Seed your source. Lab 02 Ingestion — Land Raw to Bronze Extract from Postgres and the event stream into a raw bronze layer — incrementally and idempotently. Lab 03 Transform — Silver & Gold with dbt Build the medallion transforms in dbt: clean to silver (with an SCD2 dimension), then the gold star schema your analytics run on. Lab 04 The Telemetry Firehose Handle the high-volume telemetry stream — produce it, land it, and roll it up to an hourly fact that joins your business data. The defining marketplace challenge. Lab 05 Orchestrate with Dagster Wire ingestion → dbt → rollups into one dependency-aware asset graph, scheduled and observable. Lab 06 Data Quality & Tests Add the trust layer — dbt tests, freshness checks, and a balance assertion — then break something and watch the pipeline catch it. Lab 07 The Semantic Layer & Metrics Define your metrics once — GMV, utilization, fill rate, revenue — so every consumer gets the same number. The single source of truth. Lab 08 Serve It — BI Dashboard Connect Metabase to your gold marts and build a marketplace dashboard your "stakeholders" can actually use. Lab 09 CI/CD & Observability Make it production-grade — GitHub Actions running tests on every PR, plus structured logging and run monitoring. Lab 10 Ship & Document Polish it into a portfolio piece — a clear README, an architecture diagram, one-command run instructions, and a roadmap of what to build next. Portfolio

What you'll have built

A working, documented, tested, end-to-end data platform — sources, ingestion, a medallion-modeled warehouse, a telemetry rollup, orchestration, a semantic layer, a dashboard, and CI/CD — that you can stand up with one command and explain end to end. That is exactly what a data platform engineer does, and now you'll have done it.