Start Here
This is the course that separates engineers who can only follow tutorials from engineers who can think. It teaches how computers, databases, and distributed systems actually work — the durable knowledge underneath every tool. This chapter explains why that matters for a data job, how the course is built, and how to get through it without burning out.
Why theory, for this job
It's tempting to skip foundations and go straight to the tools — learn dbt, learn Spark, get hired. It doesn't work, and here's the concrete reason: the hard parts of this job are never "which button do I click." They're "why did this pipeline run out of memory," "why is this query suddenly slow," "why did we get duplicate rows," "can we trust this number." Every one of those is a foundations question wearing a tool's clothing.
| A problem you'll hit | The foundation that explains it | Chapter |
|---|---|---|
| "The job crashed with OOM (out of memory)." | How memory works; why a join materializes rows | 02, 08 |
| "This query got 100× slower overnight." | Indexes, query planning, statistics | 03, 04 |
| "Reading this file is painfully slow." | Row vs columnar storage; compression | 03, 07 |
| "We're seeing duplicate events." | Delivery semantics; idempotency | 11 |
| "Two services disagree about the data." | Replication; consistency models | 10, 11 |
| "The API keeps timing out." | Networking, retries, backpressure | 09 |
Foundations are what make your knowledge transferable. Tools come and go — Hadoop gave way to Spark gave way to DuckDB and the lakehouse — but the reasons columnar storage is fast, joins are expensive, and distributed agreement is hard have not changed in decades and will outlast every tool you learn. This course is the highest-leverage time you'll spend.
The shape of the course
Twelve chapters, building from the smallest unit (a bit) up to the biggest idea (distributed agreement):
The order is deliberate: you can't understand a database (03–04) without algorithms (02); you can't understand distributed systems (10–11) without understanding a single machine (08) and the network between machines (09). Resist the urge to jump ahead.
The thread that connects it
To keep twelve chapters from feeling like twelve disconnected lectures, one question runs through all of them: "what actually happens, physically, when I work with data?" When you write SELECT … JOIN … GROUP BY, real bytes move from disk into memory, get hashed and sorted by real algorithms, across one or many real machines. By the end of this course you'll be able to narrate that journey — and that narration is the senior engineer's superpower.
We'll keep tying ideas back to the things you set up in Course 2 (Postgres, DuckDB, your first pipeline) so the theory always has somewhere concrete to land.
On "degree-like depth"
This course goes deeper than most bootcamps — into B-trees, query planners, memory hierarchies, and consensus algorithms. That's intentional: it's the material a computer-science degree would give you, filtered to what this job uses. But two promises:
- Always grounded. Every abstract idea is introduced through a concrete problem you'd actually face. No theory for theory's sake.
- No prerequisites beyond Courses 1–2. We build each idea from the ground up. Where math appears (mostly Big-O), it's explained plainly.
Some of this is genuinely deep. Aim for solid intuition on the first pass — enough to reason and to know what to look up. Mastery comes from returning to these chapters as you hit the real problems in later courses. Foundations is a reference you'll re-read, not a wall you clear once.
How to study this one
- Slow down. This isn't a course to skim. Budget the most time here (the schedule gives it ~2 months).
- Run the examples. Many chapters have small hands-on bits in Postgres/DuckDB/Python from Course 2 — running them turns abstract claims into things you've seen with your own eyes.
- Do every "✓ Check yourself." If you can't explain an idea simply, you don't have it yet — go back before moving on.
- It's fine to loop. Distributed systems (10–11) will make more sense after you've built pipelines in Course 4. Coming back is expected, not failure.
✓ Check yourself
- Can you explain, in one sentence, why foundations make your skills transferable across tools?
- Do you see why the chapter order goes small/concrete → big/abstract?
- Have you set aside enough time — this is the longest course?
Exercise — Predict before you learn (5 minutes)
Write down your current best guess at the answer to two of the "problems you'll hit" from the table above — e.g., why might a query get 100× slower overnight? Don't look anything up. Keep your guesses.
As you finish the relevant chapters, come back and compare. Whether you were right or wrong, the act of guessing first makes the real explanation stick far better — this is a well-studied learning effect (the "pretesting" effect), and it's why this curriculum keeps asking you to try before you're told.
Next
We start at the very bottom — what a piece of data physically is to a computer, and what really happens when you save a row to a file. → How Computers Represent Data