The Python Toolchain

Why a real toolchain, not just notebooks

Notebooks are wonderful for exploring — you'll use them forever. But production data work is code that runs unattended, on a schedule, on a machine that isn't yours. That code has to be reproducible: anyone who checks it out of git should be able to run it and get the same result. Notebooks alone don't give you that. A real project does.

The core problem a toolchain solves is dependency hell. Imagine two projects on your laptop:

If Python installs packages globally — one shared pile for the whole machine — these two projects fight. Installing pandas 2.2 for one breaks the other. You can only ever have one version, so something is always wrong.

The fix is to give each project its own private, isolated set of packages — a virtual environment. Each project carries its exact dependencies; they never collide:

Install uv

uv is a single, very fast tool that manages Python versions, virtual environments, and packages — replacing the old tangle of pip, venv, pyenv, and friends. We install it once, then let it install Python for us.

# macOS / Linux / WSL2 — one line
curl -LsSf https://astral.sh/uv/install.sh | sh

# restart your shell (or open a new terminal), then confirm
uv --version

# let uv install a modern Python for you
uv python install 3.12

Create a project

A "project" is just a folder with a few files that declare what it is and what it needs. uv init scaffolds one for you:

cd ~
uv init mini-griddp-pipeline
cd mini-griddp-pipeline
ls -a

You get a small, conventional layout. Here's what each piece is:

The heart of it is pyproject.toml — a plain text config file (in TOML format) that describes your project. Open it and you'll see something like:

[project]
name = "mini-griddp-pipeline"
version = "0.1.0"
description = "Add your description here"
requires-python = ">=3.12"
dependencies = []          # packages get listed here as you add them

Virtual environments, explained

A virtual environment (or "venv") is a private folder — .venv/ — that holds a copy of Python and only this project's packages. When a venv is active, python means "the Python in .venv" and its installed packages, nothing global. That's the isolation that makes the dependency-hell problem disappear.

The mental model: each project is a sealed box. What you install in one box never leaks into another. Two projects can hold different pandas versions because each has its own box.

# uv creates the venv for you (it also does this automatically on first 'uv add')
uv venv

# the magic command: 'uv run' runs anything INSIDE this project's venv,
# creating/syncing it first if needed — no manual "activate" step to remember
uv run python --version

Adding dependencies

To use a package, you uv add it. This installs it into the venv and records it in pyproject.toml so the project remembers it needs it. Let's add the core data tools:

uv add pandas duckdb polars

# look at what changed
cat pyproject.toml      # dependencies now lists pandas, duckdb, polars
ls                       # a new uv.lock file appeared

Two files now describe your dependencies, and the difference between them matters:

The lockfile is what makes installs reproducible across machines. When a teammate clones your repo and runs uv sync, uv reads uv.lock and installs the identical versions you had — not "whatever's newest today." Same inputs, same environment, every time. Commit pyproject.toml and uv.lock to git; never commit .venv/ (it's rebuildable and large).

A starter map of packages you'll meet constantly:

Write a real script

Time to do something useful. We'll read the data/sales.csv from Chapter 02, count the rows, and total the amount per region. First make sure the data is here:

# bring the CSV into this project (adjust the path if yours lives elsewhere)
mkdir -p data
cp ~/cli-practice/data/sales.csv data/sales.csv   # from Chapter 02

# if you skipped Chapter 02, recreate it:
cat > data/sales.csv <<'EOF'
date,region,amount
2026-06-01,us,120
2026-06-01,eu,80
2026-06-02,us,200
2026-06-02,eu,95
EOF

Now create summarize.py in the project folder with your editor (VS Code, nano, whatever you like):

import pandas as pd

# read the CSV into a dataframe (a table in memory)
df = pd.read_csv("data/sales.csv")

print(f"rows: {len(df)}")

# group by region, sum the amount column, and print each region's total
totals = df.groupby("region")["amount"].sum()
for region, total in totals.items():
    print(f"{region}: {total}")

Run it inside the project's environment:

uv run python summarize.py

rows: 4
eu: 175
us: 320

A taste of good practice

You don't need to master these yet — Courses 3 and 4 cover them fully — but a small taste now builds the right habits. Two cheap upgrades make code easier to read and trust: a docstring (what does this do?) and type hints (what goes in, what comes out?). Refactor the summary into a function:

import pandas as pd


def region_totals(path: str) -> pd.Series:
    """Read a sales CSV and return total amount per region."""
    df = pd.read_csv(path)
    return df.groupby("region")["amount"].sum()


if __name__ == "__main__":
    totals = region_totals("data/sales.csv")
    print(totals)

The -> pd.Series and path: str are type hints: they don't change how the code runs, but they document intent and let tools catch mistakes. Two tools you'll meet soon:

The --dev flag marks these as development dependencies — needed to work on the project, but not to run it in production. Keep it light for now; just know these exist and what they're for.

✓ Check yourself

• Can you explain, in one sentence, what problem a virtual environment solves?
• Do you know the difference between pyproject.toml and uv.lock — and which one makes installs reproducible?
• Could you start a fresh project, add a package, and run a script with uv run without looking it up?
• Which files do you commit to git, and which do you never commit?

uv add polars        # already added earlier? uv just confirms it's present

import polars as pl

df = pl.read_csv("data/sales.csv")

# sort by amount descending, take the first row
top = df.sort("amount", descending=True).head(1)
print(top)

uv run python top_sale.py

Next

You can build reproducible Python projects. Now let's give your pipelines real databases to talk to — spun up identically on any machine with containers. → Docker & Local Databases