Course 3 · Foundations

Foundations

The computer-science and data-systems core — taught for this job. This is the course that turns you from someone who can use tools into someone who understands why they work, so you can reason about new ones, debug the weird failures, and design rather than copy. It's the degree-equivalent material, pruned to what a data platform engineer actually uses, and grounded in concrete examples throughout.

This is the deep one

Foundations is the longest course and the one that pays off the most. Every later course — the craft, the tooling, the capstone — assumes the ideas here. Take it slowly; do the exercises. You can always reference back, and the schedule budgets the most time for exactly this course.

Chapters
00 Start Here Why a data engineer needs theory, how this course is structured, and how to study it without drowning. Read first. 01 How Computers Represent Data Bits and bytes, text encodings, binary vs text formats, and serialization (CSV, JSON, Avro, Parquet) — what a "row" really is underneath. 02 Data Structures & Algorithms for Data Big-O thinking, hash tables, trees, sorting — and the algorithms hiding inside every join, group-by, and dedup you'll ever write. 03 Inside a Database I — Storage & Indexes How tables are physically stored in pages, what a B-tree index really is, why indexes make reads fast, and the row-vs-columnar split. 04 Inside a Database II — Queries & Transactions How the engine plans and runs a query (and how to read EXPLAIN), plus transactions, ACID, and isolation levels in plain language. 05 SQL in Depth From "I can write a SELECT" to fluency — joins deeply, window functions, CTEs, set operations, and the optimization mindset. 06 Python for Data Engineering Real engineering Python — modules and packaging, type hints, error handling, generators, and testing with pytest. Beyond scripts. 07 Files & Serialization at Scale Row vs columnar files, Parquet internals, compression, partitioning, and object storage — why the file format is a performance decision. 08 Operating Systems & Resources Processes and threads, memory, disk and I/O, CPU — the resources your pipelines compete for, and the failures that follow (OOM, I/O-bound). 09 Networking & APIs Client/server, TCP and HTTP, REST APIs and auth — how data actually moves between the systems a platform stitches together. 10 Distributed Systems I — Scaling Out Why one machine isn't enough, and what changes when you use many — partitioning, replication, parallelism, and the dreaded shuffle. 11 Distributed Systems II — Consistency & Delivery CAP, consistency models, consensus (Raft/Paxos intuition), delivery semantics, and idempotency — the ideas behind every reliable pipeline.
Why this course matters

Tools change every few years; foundations don't. An engineer who knows why a columnar format is fast, why a join can blow up memory, and why exactly-once delivery is hard can pick up any new tool in days — and debug the problems that stump people who only memorized commands. That durability is the entire point of this course.