The Glossary
Every term the curriculum uses, defined in plain language and grouped by theme, each with a pointer to where it's taught in depth. Look up anything that trips you up — then keep going.
Tip
Use your browser's find (Ctrl/⌘-F) to jump straight to a term. The Taught in column points to the course and chapter where each concept is explained fully — the glossary is the quick answer; the pointer is the deep one.
Foundations & CS
| Term | Meaning | Taught in |
|---|---|---|
| ACID | Four guarantees a transaction can offer — Atomicity (all-or-nothing), Consistency, Isolation, Durability. The bar for correctness in databases that handle money. | Foundations 4 |
| API | Application Programming Interface — a defined way for one program to request data or actions from another, most often over HTTP. | Foundations 9 |
| B-tree | The balanced, sorted tree structure most database indexes use; gives fast O(log n) lookups and efficient range scans with few disk reads. | Foundations 3 |
| Big-O notation | A way to describe how an algorithm's work grows with input size (e.g. O(n) vs O(n²)) — what separates "fine" from "catastrophic" at a million rows. | Foundations 2 |
| Bit / byte | A bit is a single 0/1; a byte is 8 bits. Everything a computer stores is ultimately bytes. | Foundations 1 |
| CAP theorem | In a distributed system, when the network partitions you must choose between Consistency and Availability — you can't have both during the partition. | Foundations 11 |
| Character encoding / UTF-8 | The rules mapping characters to bytes. UTF-8 is the near-universal standard; an encoding mismatch is what produces garbled "mojibake" text. | Foundations 1 |
| Concurrency vs parallelism | Concurrency = managing many tasks that overlap in time; parallelism = literally running them at once on multiple cores. | Foundations 8 |
| Consensus (Raft/Paxos) | How a cluster of machines agrees on a single value or leader despite failures, usually by majority/quorum vote. | Foundations 11 |
| Consistency (strong vs eventual) | Strong = every read sees the latest write; eventual = reads may briefly see stale data until replicas catch up. | Foundations 11 |
| CPU-bound vs I/O-bound | Whether a job is limited by computation (CPU) or by waiting on disk/network (I/O) — it determines how you speed it up. | Foundations 8 |
| Delivery semantics | How many times a message may be processed: at-most-once (may lose), at-least-once (may duplicate — common), exactly-once (hard). | Foundations 11 |
| EXPLAIN | A SQL command that shows the query plan the engine will use — your window into why a query is slow. | Foundations 4 |
| Event-time vs processing-time | When an event actually happened vs when your system received it; bucketing by the wrong one corrupts time-window analytics. | Foundations 11 |
| Floating-point imprecision | Decimals stored in binary can't be exact (e.g. 0.1 + 0.2 ≠ 0.3), which is why money uses integers/decimals, not floats. | Foundations 1 |
| Hash table | A structure giving average O(1) lookups by key; the engine behind joins, deduplication, and group-by. | Foundations 2 |
| HTTP | The request/response protocol of the web and most APIs — methods (GET/POST…), status codes (2xx/4xx/5xx), headers, body. | Foundations 9 |
| Idempotency | An operation that's safe to run more than once — running it twice leaves the same result as running it once. The practical cure for at-least-once duplicates. | Foundations 11 |
| Index | A sorted side-structure that lets a database find rows without scanning the whole table — speeds reads, slightly slows writes. | Foundations 3 |
| Isolation level | How much concurrent transactions can affect each other (read uncommitted → serializable); a trade-off between correctness and concurrency. | Foundations 4 |
| Latency vs bandwidth | Latency = delay per request; bandwidth = volume per unit time. Many-small-requests workloads are dominated by latency. | Foundations 9 |
| Memory hierarchy | The speed/size ladder from CPU registers → cache → RAM → disk, each far slower than the last. RAM is the scarce, fast resource. | Foundations 8 |
| MVCC | Multi-Version Concurrency Control — how databases like Postgres give isolation without locking everything, by keeping row versions. | Foundations 4 |
| OLTP vs OLAP | OLTP = transactional systems (many small reads/writes, row-oriented); OLAP = analytics (big scans/aggregations, columnar). | Foundations 3 |
| OOM (out of memory) | A job crashing because it tried to hold more data in RAM than available — e.g. a huge join or loading a giant file at once. | Foundations 8 |
| Partitioning / sharding | Splitting data across machines (or files) by a key so each handles a slice; a bad key causes skew. | Foundations 10 |
| Process vs thread | A process is an isolated running program with its own memory; threads share memory within a process. | Foundations 8 |
| Query planner / optimizer | The database component that turns declarative SQL into an execution plan, using statistics to estimate costs. | Foundations 4 |
| Replication | Keeping copies of data on multiple machines for durability and read scaling; copies must be kept in sync. | Foundations 10 |
| REST | A common style of HTTP API — resources at URLs, returning JSON, with pagination and rate limits. | Foundations 9 |
| Row-oriented vs columnar | Row stores keep a record's fields together (good for OLTP); columnar stores keep each column together (good for analytics scans + compression). | Foundations 3 |
| Serialization | Turning in-memory data into bytes for storage/transfer (and back = deserialization) — e.g. to CSV, JSON, or Parquet. | Foundations 1 |
| Shuffle | Moving data between machines so related rows meet (for a cross-partition join or group-by) — the expensive part of distributed processing. | Foundations 10 |
| TCP / DNS | TCP = the reliable, ordered transport under most network traffic; DNS = the system translating names into IP addresses. | Foundations 9 |
| Watermark | In streaming, a moving marker of "event-time progress" used to decide when a time window is complete and how to handle late data. | Foundations 11 |
Storage & Files
| Term | Meaning | Taught in |
|---|---|---|
| Avro | A compact, schema-based binary row format common in streaming. | Foundations 7 |
| ClickHouse | A fast columnar database built for high-volume time-series/analytics — used for telemetry firehoses where a general warehouse is too costly. | Foundations 7; Systems Design 5 |
| Compaction | Merging many small files into fewer larger ones to fix the small-files problem and keep queries fast. | Foundations 7 |
| Compression (Snappy/Zstd/gzip) | Shrinking data on disk; a CPU-vs-size trade-off. Columnar data compresses especially well. | Foundations 7 |
| CSV / JSON | Human-readable text formats — simple and universal, but larger and slower than binary/columnar for analytics. | Foundations 1, 7 |
| Data lake | Cheap, open file storage (usually object storage) holding raw and processed data of any shape. | The DE Craft 8 |
| Data warehouse | A system optimized for analytical queries over structured, modeled data (e.g. Snowflake, BigQuery, DuckDB). | The DE Craft 8 |
| Lakehouse | A design combining the lake's cheap open storage with the warehouse's tables, ACID, and schema. | The DE Craft 8 |
| MinIO | An open-source, S3-compatible object store — the local stand-in for cloud object storage in this curriculum. | Tooling 4 |
| Object storage (S3/GCS) | Cheap, durable, effectively infinite cloud file storage addressed by key — the backbone of data lakes. | Foundations 7; Tooling 4 |
| Parquet | The standard columnar file format — typed, compressed, with row groups and per-column statistics enabling data-skipping. | Foundations 7 |
| Partition pruning | Skipping whole files/partitions a query doesn't need (e.g. other dates), thanks to a partitioned layout. | Foundations 7; The DE Craft 8 |
| Predicate / projection pushdown | Reading only the rows (predicate) and columns (projection) a query needs, using file statistics — making columnar scans cheap. | Foundations 7 |
| Row group | A horizontal chunk of a Parquet file holding column data + min/max stats for that chunk. | Foundations 7 |
| Small-files problem | Too many tiny files (often from streaming) making queries slow; fixed by compaction. | Foundations 7 |
| Table format (Iceberg / Delta / Hudi) | A layer over Parquet files that adds database-like features — ACID transactions, schema evolution, and time travel. | The DE Craft 8 |
| Time travel | Querying a table as it existed at an earlier point, via a table format's version history. | The DE Craft 8 |
Data Modeling
| Term | Meaning | Taught in |
|---|---|---|
| Conformed dimension | A dimension shared across multiple fact tables/marts so metrics join consistently everywhere. | The DE Craft 2; Systems Design 6 |
| Dimension table | Descriptive context (who/what/where) — wide, denormalized tables like dim_customer, dim_gpu. | The DE Craft 2 |
| Dimensional modeling | Designing analytics schemas as facts surrounded by dimensions (the star schema), optimized for understandable, fast queries. | The DE Craft 2 |
| ERD | Entity-Relationship Diagram — a picture of entities, their attributes, and how they relate. | The DE Craft 1 |
| Fact table | The events/measurements at the center of a star schema (e.g. fct_rentals), with measures and keys to dimensions. | The DE Craft 2 |
| Grain | What one row of a fact table represents (e.g. "one rental-hour") — the first and most important modeling decision. | The DE Craft 2 |
| Measure (additive / semi / non-additive) | A numeric fact column; additive sums across all dimensions (revenue), non-additive doesn't (a percentage). | The DE Craft 2 |
| Medallion architecture | Layering transforms as bronze (raw) → silver (cleaned) → gold (business-ready) for debuggability and reuse. | The DE Craft 4 |
| Normalization (1NF/2NF/3NF) | Structuring relational tables to remove redundancy and update anomalies — ideal for OLTP sources. | The DE Craft 1 |
| One Big Table (OBT) | A wide, fully denormalized table; sometimes used for ML features or simple analytics instead of a star schema. | Systems Design 6 |
| Point-in-time correctness | Joining a fact to the dimension version that was valid at the fact's timestamp (not the latest) — the basis of trustworthy history. | The DE Craft 3 |
| Primary key / foreign key | A primary key uniquely identifies a row; a foreign key links a row to a primary key in another table. | The DE Craft 1 |
| Slowly-changing dimension (SCD) | How to handle dimension attributes that change: Type 1 overwrites, Type 2 keeps history as new rows, Type 3 keeps one prior value. | The DE Craft 3 |
| Star schema | A central fact table joined to surrounding dimension tables — the classic analytics model. | The DE Craft 2 |
| Surrogate vs natural key | A surrogate key is a system-generated id; a natural key is a real-world identifier. SCD2 relies on surrogate keys. | The DE Craft 1, 3 |
Pipelines & Ingestion
| Term | Meaning | Taught in |
|---|---|---|
| Backfill | Re-running a pipeline over past dates/data — to fix a bug or load history; safe only if transforms are idempotent. | The DE Craft 5; Systems Design 7 |
| Batch vs streaming | Processing data on a schedule (batch) vs as it arrives (streaming). Most pipelines are batch; streaming adds low latency and complexity. | The DE Craft 5, 9 |
| Change Data Capture (CDC) | Capturing every insert/update/delete from a source database as a stream of change events, rather than re-snapshotting. | The DE Craft 6 |
| Data contract | An explicit, versioned agreement with a source team about a dataset's schema, semantics, and SLAs — pushing quality "left." | The DE Craft 11; Systems Design 1 |
| Dead-letter queue | A holding place for messages/records that failed processing, so the pipeline keeps running and bad data can be inspected. | The DE Craft 5; Systems Design 4 |
| Debezium | A popular open-source tool that reads a database's write-ahead log to produce CDC event streams. | The DE Craft 6 |
| ETL vs ELT | ETL transforms data before loading; ELT loads raw then transforms in the warehouse (the modern default). | The DE Craft 5 |
| High-watermark / incremental load | Pulling only records newer than the last one seen (tracking a max id/timestamp), instead of re-loading everything. | The DE Craft 5 |
| Schema registry | A central store of event schemas (e.g. Avro) that enforces compatibility as producers evolve, so consumers don't break. | The DE Craft 6; Systems Design 4 |
| Upsert / merge | Insert-or-update on a key — the common way to make a load idempotent. | The DE Craft 5 |
| Write-ahead log (WAL) | The durable log a database writes before applying changes; log-based CDC tails it to capture every change. | The DE Craft 6 |
Transformation & Orchestration
| Term | Meaning | Taught in |
|---|---|---|
| Asset (Dagster) | A declared data object (a table/file) Dagster produces; you define assets and their dependencies and it builds the graph. | The DE Craft 10 |
| DAG | Directed Acyclic Graph — the dependency graph of pipeline steps (or dbt models), defining what runs before what. | The DE Craft 7, 10 |
| dbt | A tool for version-controlled, tested, documented SQL transformations; models are SELECTs wired into a DAG via refs. | The DE Craft 7 |
| dbt test | A built-in or custom assertion about data (not_null, unique, relationships, accepted_values…) run with the pipeline. | The DE Craft 7, 11 |
| Freshness SLA | A target for how recent data must be (e.g. "loaded within 2 hours"); breaching it triggers an alert. | The DE Craft 11; Systems Design 7 |
| Lineage | The map of which datasets feed which — used for impact analysis ("if this breaks, what's affected?"). | The DE Craft 7; Systems Design 10 |
| Materialization (view/table/incremental) | How dbt builds a model — as a view, a full table, or an incremental table that only processes new rows. | The DE Craft 7 |
| Model | In dbt, a SELECT statement in a .sql file that becomes a view or table; the unit of transformation. | The DE Craft 7 |
| Orchestrator (Airflow/Dagster/Prefect) | A tool that schedules pipeline steps, runs them in dependency order, retries failures, and gives visibility. | The DE Craft 10 |
| ref() / source() | dbt functions: ref() references another model (building the DAG); source() declares a raw input table. | The DE Craft 7 |
| Schedule / sensor | A schedule runs a pipeline on a cron; a sensor triggers it when upstream data is ready (data-aware). | The DE Craft 10 |
| Snapshot (dbt) | A dbt feature that captures changes to a source over time — a built-in way to implement SCD2. | The DE Craft 3, 7 |
Streaming
| Term | Meaning | Taught in |
|---|---|---|
| Backpressure | When a consumer can't keep up with a producer; the system must buffer or slow down rather than drop data. | The DE Craft 9; Systems Design 4 |
| Consumer / consumer group | A consumer reads events from a topic; a consumer group splits a topic's partitions across members for parallel reading. | The DE Craft 9 |
| Kafka / Redpanda | Durable, distributed logs for streaming events. Redpanda is a Kafka-compatible engine; this curriculum uses it locally. | The DE Craft 9 |
| Offset | A consumer's bookmark — the position in a partition it has read up to; enables replay and at-least-once delivery. | The DE Craft 9 |
| Producer | A program that writes (publishes) events to a topic. | The DE Craft 9 |
| Topic / partition | A topic is a named stream of events; it's split into partitions, which are ordered and enable parallelism. | The DE Craft 9 |
| Window (tumbling/sliding) | A time bucket for aggregating a stream (e.g. "events per minute"); keyed on event-time. | Foundations 11; The DE Craft 9 |
| The log | The core streaming abstraction — an append-only, ordered sequence of events. | The DE Craft 9 |
Serving & Semantic
| Term | Meaning | Taught in |
|---|---|---|
| BI / dashboard | Business Intelligence tools (Metabase, Looker…) that turn modeled data into charts and dashboards for people. | Capstone Lab 08; Systems Design 8 |
| Data API | A service that serves modeled data to applications (e.g. feeding scores back into a product) at low latency. | Systems Design 8 |
| Data catalog | A searchable inventory of datasets — schemas, owners, freshness, lineage — so people can find and trust data. | Systems Design 9 |
| GMV (example metric) | Gross Merchandise Value — total transaction value flowing through a marketplace; a typical headline metric. | Systems Design 8; Capstone Lab 07 |
| Metric | A precisely-defined business measure (GMV, utilization, churn) with a fixed formula and grain. | Systems Design 8; Capstone Lab 07 |
| Reverse ETL | Pushing modeled warehouse data back into operational tools (CRM, ops consoles) so teams act on it. | Systems Design 8 |
| Self-serve analytics | Letting non-engineers answer their own data questions safely, via catalogs, certified datasets, and a semantic layer. | Systems Design 9 |
| Semantic layer / metrics layer | A single, version-controlled definition of dimensions and metrics that compiles to SQL — so every consumer (and AI) gets the same number. | Systems Design 8; Capstone Lab 07 |
Quality & Governance
| Term | Meaning | Taught in |
|---|---|---|
| Audit log | A record of who accessed or changed what — required for compliance and security investigations. | Systems Design 11 |
| Data observability | Automated monitoring for data problems you didn't write explicit tests for — freshness, volume, schema, and distribution anomalies. | The DE Craft 11; Tooling 7 |
| Data quality dimensions | The axes of "is the data right?": freshness, volume, schema, validity, uniqueness, referential integrity. | The DE Craft 11; Systems Design 10 |
| Data residency | Rules about which country/region data may be stored or processed in — driven by law and export controls. | Systems Design 11 |
| Data SLO | A Service-Level Objective for data (e.g. "99% of days, the table is fresh by 6am") — a measurable reliability target. | The DE Craft 11; Systems Design 10 |
| FinOps / cost governance | Managing cloud/data spend — chargeback, query-cost limits, auto-suspend — so a platform doesn't quietly get expensive. | Systems Design 11; Tooling 4 |
| Governance | The systems and policies that make data safe to open up — access control, classification, audit — as an enabler, not a blocker. | Systems Design 11 |
| PII | Personally Identifiable Information — sensitive personal data (names, emails, KYC) needing extra protection. | Systems Design 11 |
| RBAC / ABAC | Role-Based vs Attribute-Based Access Control — two models for deciding who can see/do what, plus row/column-level security. | Systems Design 11 |
| Tokenization / masking | Replacing sensitive values with tokens (or hiding them in query results) to limit the blast radius of PII. | Systems Design 11 |
Tooling & Cloud
| Term | Meaning | Taught in |
|---|---|---|
| CI/CD | Continuous Integration / Continuous Deployment — automatically testing every change and deploying safely. | Tooling 5 |
| Container / image | An image is a packaged template of software + dependencies; a container is a running instance of one. | Orientation 5; Tooling 2 |
| Docker / Compose | Docker packages software into containers; Compose defines and runs a multi-service stack from one file. | Orientation 5; Tooling 2–3 |
| Dockerfile | A recipe for building a Docker image — base image, dependencies, code, and the command to run. | Tooling 2 |
| git: branch / commit / PR / merge conflict | Commit = a saved change; branch = an isolated line of work; pull request = a proposed merge + review; conflict = two edits to the same lines needing manual resolution. | Orientation 3; Tooling 1 |
| GitHub Actions | GitHub's CI/CD system — YAML workflows that run on events (e.g. run dbt tests on every pull request). | Tooling 5 |
| IAM | Identity & Access Management — cloud users, roles, and policies; the main security surface (least privilege). | Tooling 4 |
| IaaS / PaaS / SaaS | Cloud service models — Infrastructure, Platform, and Software as a Service — differing in how much the provider manages. | Tooling 4 |
| Infrastructure as Code (IaC) | Defining infrastructure in version-controlled code (e.g. Terraform) instead of clicking in a console. | Tooling 6 |
| Observability (logs/metrics/traces) | The three signals for understanding a running system — what happened, numbers over time, and a request's path. | Tooling 7 |
| Region / availability zone | A cloud region is a geographic location; zones are isolated datacenters within it for redundancy. | Tooling 4 |
| SLA vs SLO | An SLA is a promise to customers (often contractual); an SLO is an internal target you measure against. | Tooling 7 |
| Terraform / state | An IaC tool using declarative HCL; its state file tracks what it created so plan/apply can reconcile reality. | Tooling 6 |
| Volume (Docker) | Persistent storage attached to a container so data survives restarts. | Tooling 2–3 |
AI & ML for Data
| Term | Meaning | Taught in |
|---|---|---|
| Drift | When live data or model performance moves away from what a model was trained on — a signal to retrain. | Systems Design 12 |
| Embedding / vector store | A numeric vector representing meaning (of text/images); a vector store indexes them for similarity search and RAG. | Systems Design 12–13 |
| Feature / feature store | A feature is a model input derived from data; a feature store serves the same feature consistently for training (offline) and inference (online). | Systems Design 12 |
| Grounding | Constraining an AI's output to trusted context (e.g. the semantic layer) so answers are correct-by-construction, not guessed. | Systems Design 13 |
| Label leakage | Accidentally training on information that wouldn't be known at prediction time, inflating accuracy and breaking in production. | Systems Design 12 |
| Offline/online parity (training-serving skew) | The bug where features computed for training differ from those at serving; a feature store prevents it. | Systems Design 12 |
| RAG | Retrieval-Augmented Generation — fetching relevant context and feeding it to an LLM so it answers from real data. | Systems Design 13 |
| Text-to-SQL | Turning a natural-language question into SQL; accurate only when grounded on a semantic layer, not a raw schema. | Systems Design 13 |
| Training data / point-in-time | The labeled dataset a model learns from, assembled with only the information available at each prediction time. | Systems Design 12 |
Career
| Term | Meaning | Taught in |
|---|---|---|
| ATS | Applicant Tracking System — software that screens resumes by keywords before a human sees them. | Career 3 |
| Hiring funnel | The stages from application → resume screen → recruiter → technical screens → onsite → offer; each filters candidates out. | Career 0 |
| Referral | An introduction from someone inside a company — by far the highest-yield application channel. | Career 1, 3 |
| STAR method | A structure for behavioral answers — Situation, Task, Action, Result. | Career 5 |
| System design interview | A round where you design a pipeline/platform out loud — exactly what this curriculum (and the capstone) prepares you for. | Career 5; Systems Design (all) |
| Take-home | An assignment done on your own time; graders reward clean code, tests, a README, and sensible modeling. | Career 5 |
Keep it open
Every term above links back to the course and chapter that teaches it in full. Use this page to stay unblocked in the moment, and the pointers when you want the depth. Next door are the cheat sheets for syntax and decisions.