Start Here
This course turns your laptop into a data-engineering workstation and gives you the mental model to use it. Before we install anything, this chapter maps the toolchain — what each tool is, why it exists, and how the pieces connect — so the setup chapters feel like assembling something you understand, not following magic incantations.
Why a whole course on setup
Most beginners try to skip this part and pay for it later — stuck for hours on an environment problem that has nothing to do with data engineering. Professionals are fast in their environment: they move around the filesystem without thinking, version every change in git, reproduce any setup with a single command. That fluency is a force multiplier for everything that follows.
So we treat the environment as a first-class skill. By the end of this course you'll have the same basic toolkit a working data engineer uses every day, and — more importantly — you'll understand what each piece does.
A theme starts here and never leaves: everything is code, and everything is reproducible. Your environment, your pipelines, your infrastructure — all defined in files, all in git, all rebuildable from scratch. "It works on my machine" is the problem this whole discipline is organized to eliminate.
The toolchain, mapped
Here's everything this course installs and why. Don't install anything yet — just get the lay of the land.
| Tool | What it is, in one line | Why a data engineer needs it | Chapter |
|---|---|---|---|
| The shell / CLI | A text interface to drive your computer | Every server, container, and data tool is driven from here | 02 |
| git + GitHub | Version control + a place to host it | All code — pipelines, configs, infra — lives in version control | 03 |
| Python + uv | The language + a fast environment manager | The lingua franca for pipelines, glue, and data tools | 04 |
| Docker + Compose | Run software in isolated, reproducible containers | Spin up databases and tools identically on any machine | 05 |
| Postgres | A classic relational (OLTP) database | Stands in for the source systems that produce data | 05 |
| DuckDB | A fast in-process analytics (OLAP) database | Your local "data warehouse" for analysis and transforms | 05 |
How they fit together — the shape you'll assemble:
A note on your operating system
Data engineering happens on Linux servers, so we work in a Linux-like environment everywhere:
- macOS — already Unix-like; you're good. We'll use the built-in Terminal and install a package manager (Homebrew).
- Linux — you're in the native environment; perfect.
- Windows — install WSL2 (Windows Subsystem for Linux) and do everything inside it. Don't fight the Windows shell for this work; WSL2 gives you a real Ubuntu. Each hands-on chapter notes any Windows-specific step.
Before Chapter 02, open PowerShell as Administrator and run wsl --install, reboot, and finish the Ubuntu setup. From then on, "open a terminal" means "open your WSL2 Ubuntu terminal." This one step prevents a hundred future headaches.
How the hands-on chapters work
Starting in Chapter 02, chapters are labs. Each one has a consistent shape so you always know what to do:
| Element | Looks like | What to do |
|---|---|---|
| Setup box | A callout listing what to install/have running first | Complete it before the rest of the chapter |
| Runnable blocks | Code blocks with a filename/shell caption | Type them into your terminal/editor and run |
| ✓ You should see | A callout with expected output | Confirm your result matches before continuing |
| ✓ Check yourself + Exercise | A checklist + a collapsible exercise | Do it from memory; reveal the solution after |
If a "You should see" doesn't match what you got, stop and fix it there — debugging the small step now is far easier than discovering it broke three chapters later. (Course 1's "when you get stuck" guidance applies throughout.)
✓ Check yourself
- Can you name the six tools this course installs and say what each is for?
- Do you know which terminal you'll be using (and, if on Windows, is WSL2 installed)?
- Do you understand the "everything is code and reproducible" principle?
Exercise — Confirm you have a working terminal (1 minute)
Open your terminal (WSL2 Ubuntu on Windows) and run:
uname -s # prints the OS kernel name
whoami # prints your usernameYou should see something like Darwin (macOS) or Linux (Linux/WSL2), then your username. If uname isn't found, you're likely in the Windows shell rather than WSL2 — switch terminals and try again. That's it — you have a working shell, and Chapter 02 takes it from here.
Next
Tools make more sense once you can see the whole machine they serve. Next we install the mental model — the one picture of how data moves through a platform that the rest of the curriculum hangs on. → The Data Platform Mental Model