Course 2 · Hands-on

Orientation & Setup

The first hands-on course. We install the mental model that makes everything else make sense, then build the working environment every data engineer lives in — the command line, git, a real Python toolchain, and Docker running a local Postgres and DuckDB — and finish by running your first end-to-end data pipeline. From here on, you learn by doing.

Before you start

You'll need a computer (macOS, Linux, or Windows with WSL2), a stable internet connection, and about 10 GB of free disk. Everything we install is free and open-source. Coming from Course 1? You should already have your mini-griddp repo started — we'll build into it here.

Chapters

00 Start Here What this course sets up and why each piece matters. A map of the toolchain before we install it. 01 The Data Platform Mental Model The single picture that makes the whole field click — data's journey from source to decision, the layers it passes through, and the vocabulary you'll keep meeting. 02 The Command Line The shell every engineer lives in — navigating, files, pipes and redirection, and the handful of commands that do 90% of the work. Hands-on. 03 Git & GitHub Version control from zero — commits, branches, the everyday workflow, and pushing your companion repo to GitHub. Why everything in this job is code. Hands-on. 04 The Python Toolchain A real project setup — virtual environments with uv, project structure, installing packages, and writing a script that reads data. Beyond notebooks. Hands-on. 05 Docker & Local Databases Containers in plain language, then Docker Compose to run a local Postgres and DuckDB — the two databases you'll use for the rest of the curriculum. Hands-on. 06 Your First Pipeline Tie it together: extract rows from Postgres, transform them in Python/DuckDB, and load the result — your first real extract-transform-load, with checkpoints at every step. Capstone of the course

What you'll have built

By the end of this course your laptop has a working data-engineering environment, and your mini-griddp repo contains your first pipeline:

┌─────────────┐ extract ┌──────────────┐ transform ┌──────────────┐ │ Postgres │───────────────▶│ Python + │──────────────▶│ DuckDB │ │ (source DB, │ (read rows) │ pandas/polars │ (clean, │ (your local │ │ in Docker) │ │ │ reshape) │ warehouse) │ └─────────────┘ └──────────────┘ └──────────────┘ ▲ │ │ all running locally, defined as code in your git repo ▼ └───────────────── you can tear it down and rebuild it anytime ─┘

It's small — but it's the same shape as the platforms you'll design later. Everything after this course just adds rigor, scale, and tools to this exact picture.