Cheat Sheets
Compact quick-reference cards for the things you look up constantly while building — SQL patterns, the everyday commands, dbt and Dagster, the stack map, and the decisions you'll make again and again.
SQL patterns
The patterns that come up in real work and in interviews. (Depth: Foundations, ch. 5.)
-- INNER keeps matches; LEFT keeps all left rows (NULLs where no match)
SELECT c.name, r.gpu_id
FROM customers c
LEFT JOIN rentals r ON r.customer_id = c.customer_id;
-- group + filter groups with HAVING (WHERE filters rows, HAVING filters groups)
SELECT gpu_id, SUM(hours) AS total_hours
FROM rentals
GROUP BY gpu_id
HAVING SUM(hours) > 100;SELECT
customer_id,
started_at,
amount,
-- running total per customer over time
SUM(amount) OVER (PARTITION BY customer_id ORDER BY started_at) AS running_total,
-- rank rentals within each customer (1 = most recent)
ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY started_at DESC) AS rn,
-- previous row's value
LAG(amount) OVER (PARTITION BY customer_id ORDER BY started_at) AS prev_amount
FROM rentals;
-- TOP-N PER GROUP: rank, then filter in an outer query
SELECT * FROM (
SELECT *, ROW_NUMBER() OVER (PARTITION BY gpu_id ORDER BY amount DESC) AS rn
FROM rentals
) t
WHERE rn <= 3;
-- DEDUP: keep one row per key
SELECT * FROM (
SELECT *, ROW_NUMBER() OVER (PARTITION BY rental_id ORDER BY loaded_at DESC) AS rn
FROM bronze_rentals
) t WHERE rn = 1;WITH monthly AS (
SELECT date_trunc('month', started_at) AS month, SUM(amount) AS revenue
FROM rentals GROUP BY 1
)
SELECT month, revenue,
revenue - LAG(revenue) OVER (ORDER BY month) AS mom_change
FROM monthly ORDER BY month;Shell, git & Docker
The everyday commands. (Depth: Orientation ch. 2–3, Tooling ch. 1–2.)
pwd; ls -la; cd path # where am I / what's here / move
cat f | grep "err" | wc -l # pipe: read → filter → count
head -n 100 big.csv > sample.csv # first 100 rows → new file
find . -name "*.parquet" # find files by pattern
grep -r "TODO" . # recursive text searchgit status # what changed
git switch -c my-feature # new branch
git add -A && git commit -m "msg"
git push -u origin my-feature # then open a Pull Request
git switch main && git pull # sync main
git merge my-feature # (or merge via the PR)docker build -t myimg . # build an image from a Dockerfile
docker run --rm myimg # run a container
docker compose up -d # start the whole stack (detached)
docker compose ps # what's running
docker compose logs -f svc # follow a service's logs
docker compose exec postgres psql -U postgres # shell into a service
docker compose down # stop it alldbt & Dagster
(Depth: The DE Craft ch. 7 & ch. 10.)
-- models/marts/fct_revenue.sql
SELECT gpu_id, SUM(hours * price_per_hour) AS revenue
FROM {{ ref('stg_rentals') }}
GROUP BY 1models:
- name: fct_revenue
columns:
- name: gpu_id
tests: [not_null, unique]
- name: revenue
tests:
- dbt_utils.accepted_range: {min_value: 0}# dbt: dbt build (run models + tests)
# dbt test (just tests)
# dbt docs generate && dbt docs serve
# Dagster — an asset with a dependency
from dagster import asset
@asset
def bronze_rentals() -> None:
... # extract from Postgres
@asset(deps=[bronze_rentals])
def fct_revenue() -> None:
... # transform
# run the UI: dagster devThe stack at a glance
Each platform layer, the local tool this curriculum teaches it on, and a common production equivalent.
| Layer | Does | Local (this curriculum) | Common in production |
|---|---|---|---|
| Source | Produces data | Postgres + a generator | App databases, SaaS, events |
| Ingest | Moves data in | Python scripts, Redpanda | Fivetran, Airbyte, Kafka, Debezium |
| Store | Holds data | DuckDB + Parquet (lakehouse) | Snowflake / BigQuery / Iceberg on S3 |
| Transform | Cleans & models | dbt | dbt, Spark |
| Orchestrate | Schedules & orders | Dagster | Dagster, Airflow |
| Serve | Delivers to consumers | Metabase + a semantic layer | Looker, Tableau, semantic layers |
| Govern / operate | Quality, access, monitoring | dbt tests, logs, Dagster UI | Catalogs, observability tools, IAM |
Decision heuristics
The recurring "which do I pick?" calls, with a one-line rule of thumb.
| Decision | Rule of thumb |
|---|---|
| Batch vs streaming | Default to batch; reach for streaming only when you genuinely need low latency. |
| Normalize vs dimensional | Normalize OLTP sources (avoid anomalies); model analytics dimensionally (fast, understandable reads). |
| ETL vs ELT | Prefer ELT — land raw, then transform in the warehouse with SQL/dbt. |
| Warehouse vs lakehouse | A warehouse is simpler; choose a lakehouse for open formats, scale, and serving BI + ML from one copy. |
| Row vs columnar storage | Row for transactional point lookups/writes; columnar for analytical scans and aggregation. |
| SCD Type 1 vs Type 2 | Type 1 if you don't need history; Type 2 whenever "what was true at the time?" matters. |
| When to add an index | Add one when a frequent query filters/joins on a column and reads outweigh writes. |
| Full vs incremental load | Incremental once the table is large; full loads are fine while data is small. |
| When to use CDC | When you need every change (including deletes) and full history, not just the latest snapshot. |
| View vs table vs incremental (dbt) | View for cheap/fresh; table for reused/expensive; incremental for large append-mostly facts. |
Rules of thumb & mental models
- Match the ingest pattern to the source's shape — CDC for mutable DBs, streaming for events, batch for APIs. (Systems Design 1, 4)
- Declare the grain first — before modeling a fact table, state what one row means. (The DE Craft 2)
- Never route the firehose through the warehouse — high-volume telemetry goes to a time-series store + rollups. (Systems Design 5, 14)
- Idempotency makes retries safe — design loads so running twice equals running once. (Foundations 11)
- The semantic layer is the single source of truth — define each metric once; it's also what makes AI text-to-SQL accurate. (Systems Design 8, 13)
- Quality is set at the source — the cheapest place to fix data is in the producing system, via contracts. (Systems Design 1)
- The memory hierarchy spans orders of magnitude — RAM is vastly faster than disk, and the scarce resource; most OOMs are "tried to hold too much at once." (Foundations 8)
- Everything is code, everything is reproducible — pipelines, config, and infrastructure all in git. (Tooling, all)
- Build only what's differentiating — buy storage, CDC, and warehouses; build the glue and golden paths. (Systems Design 3)
You've reached the end
That's the whole reference — and the end of the Data Platform curriculum. Eight courses took you from "I can write a basic query and a small script" to a hired, ramping data platform engineer who has designed, built, and operated a real platform.
Keep this glossary and these cheat sheets open as you build; nobody holds it all in their head. When you want the senior, big-picture view of how every piece fits together at scale, the Data Platform Systems Design reference is now yours to read as a peer.
You did the hard part — you kept going. Head back to the curriculum home any time to revisit a course, and go build something real. 🎉