Operating Systems & Resources

Why a data engineer needs OS basics

You will spend a surprising amount of this career staring at a job that died for no obvious reason. The logs say Killed, or OOMKilled, or the run just hangs at 100% and never finishes. The instinct is to blame the tool — "Spark is flaky," "pandas is slow." It almost never is. What's actually happening is that your program, like every program, is competing for three physical resources that a referee called the operating system (OS) hands out: CPU (the chips that do the computing), memory / RAM (fast scratch space), and disk (slow permanent storage). When you run out of one, the OS does something abrupt, and your pipeline dies.

A program is a file of instructions; a process is that program actually running, with its own slice of memory the OS grants it. Your pipeline is just a process (or a few). The OS's whole job is to share one machine's resources across many processes — yours, the database's, the browser you forgot to close — and to lie to each of them that it has the machine to itself. Knowing where that lie breaks down is the difference between guessing and diagnosing.

Processes, threads & the GIL

A process is an isolated running program with its own private memory. A thread is a stream of execution inside a process; one process can have many threads that share its memory. Roughly: processes are separate houses, threads are people in the same house sharing a kitchen.

Two words people use loosely but that matter here. Concurrency means juggling several tasks so they all make progress — like one cook starting the pasta, then chopping veg while it boils. Parallelism means literally doing several tasks at the same instant — several cooks, several stoves. You can have concurrency on a single CPU core (it switches between tasks fast); you only get true parallelism with multiple cores.

This brings us to a famous Python wrinkle. CPython (the standard Python) has a Global Interpreter Lock (GIL): a lock that lets only one thread run Python bytecode at a time, even on a machine with many cores. So spinning up ten threads to do ten heavy calculations does not give you ten-way parallelism — they take turns holding the GIL.

That's why data engineers usually don't reach for threads to speed up CPU-bound work. Instead they reach for multiprocessing (separate processes, each with its own interpreter and its own GIL, so they genuinely run in parallel) or for a distributed/columnar engine (DuckDB, Spark, Polars) that does its heavy number-crunching in compiled C/C++/Rust code which releases the GIL and uses all your cores for you.

Memory & out-of-memory

Not all storage is equal. There's a memory hierarchy: a few tiers, each one dramatically faster but smaller and pricier than the one below it. Every byte your code touches lives somewhere on this ladder, and where it lives decides how fast your job runs.

The key takeaway: RAM is fast but small. A laptop might have 16 GB of RAM and 1 TB of disk — so RAM is ~60× scarcer. Yet RAM is where data has to be for the CPU to actually work on it. Anything you want to compute on must first be loaded from slow disk into fast RAM. That loading is exactly where pipelines die.

Out of memory (OOM) is what happens when a process tries to use more RAM than is available. The OS can't grant it, so it kills the process (on Linux, the "OOM killer" picks a victim and terminates it — your log shows Killed). There's no graceful warning; the process just vanishes.

Why do data jobs trigger this so reliably? Because so many operations materialize data — pull the whole thing into RAM at once:

• pandas.read_csv("huge.csv") loads the entire file into RAM as a DataFrame — and the in-memory form is often 2–10× larger than the file on disk (Python objects, indexes, type overhead). A 10 GB CSV can need 30–80 GB of RAM.
• A big JOIN may build a hash table of one whole side in RAM before matching rows; a wide one can explode row counts.
• A GROUP BY or a sort may need every group, or the whole dataset, held in memory at once.

The fixes are patterns you've already met. Stream / chunk the data (Chapter 06) so you only ever hold a slice in RAM at a time — process and discard, never materialize the whole file. Or push the work down to the database/engine (DuckDB, Postgres, Spark), which is built to spill to disk and process more-than-RAM datasets without falling over. The rule of thumb: if the data doesn't fit in RAM, don't load it all into RAM — let a tool that streams or spills do the heavy lifting.

Disk & I/O

I/O ("input/output") is the work of moving bytes to and from disk (or the network). It's slow compared to CPU and RAM, so how you read matters enormously.

Two access patterns: a sequential read grabs bytes in order, in one continuous sweep (like playing a record start to finish); a random read jumps all over the disk to fetch scattered bytes (like skipping to track 7, then 2, then 19). Sequential is far faster — the disk (and the OS's read-ahead) is optimized for it. On old spinning HDDs, random reads meant physically moving a head and waiting for the platter to spin, which is glacial; SSDs have no moving parts and are much better at random access, but sequential still wins because it reads big contiguous blocks.

This is exactly why the columnar formats from Chapter 07 (Parquet) are fast: storing each column together lets a query read just the columns it needs, in long sequential scans, skipping the rest. Row formats force you to read everything to get one column.

The most useful diagnostic question you can ask about any slow job: is it I/O-bound or CPU-bound? An I/O-bound job spends most of its time waiting for bytes (reading a huge file, calling a slow API) — the CPU sits mostly idle. A CPU-bound job spends most of its time computing (heavy math, parsing, compression) — the CPU is pegged near 100%. The cure is different for each, so naming it first saves you from optimizing the wrong thing.

CPU & parallelism

A modern CPU has multiple cores — independent units that can each run instructions at the same time. A 4-core laptop can genuinely do four things at once; an 8-core, eight. Many tools, left alone, use only one core, which means a job can crawl while seven cores sit idle.

This is where columnar, vectorized engines shine. DuckDB, Spark, and Polars process data in batches (vectors) of column values using compiled code, and they automatically split that work across all your cores. So the same group-by that pins a single-threaded pandas script to one core can run several times faster on DuckDB just by using the whole CPU — parallelism within one machine, for free, without you writing any threading code. (Spark extends the same idea across many machines, which is Chapters 10–11.)

Observing resources in practice

You don't have to guess which resource is the problem — you can watch. top is built into every Unix system; htop is a friendlier, colorful version (install with brew install htop on macOS or sudo apt install htop on Linux). Both show, live, which processes are eating CPU and memory.

top            # live process list; press q to quit
# or, nicer:
htop           # arrow keys to scroll, F10 or q to quit

The columns that matter:

Now let's create a job to watch, so you can see memory grow with your own eyes. Open two terminals: run htop (or top) in one, and this Python in the other.

import time

# Allocate ~100 MB chunks and hold onto them, so RAM keeps climbing.
chunks = []
for i in range(1, 21):
    # a list of 12 million references ≈ ~100 MB in RAM (8 bytes each)
    chunks.append([0] * 12_000_000)
    print(f"allocated chunk {i} (~{i * 100} MB held)")
    time.sleep(1)          # pause so you can watch it in top/htop

print("done holding memory; sleeping 10s before releasing")
time.sleep(10)

python3 memory_demo.py

✓ Check yourself

Answer from memory before opening each one.

1. To speed up a CPU-heavy parser written in pure Python, a teammate runs it in 8 threads on an 8-core machine. top shows the process at about 100% CPU rather than 800%, and the job is no faster. What is going on, and what are the two ways to get real parallelism?

   Show answer

   CPython's Global Interpreter Lock lets only one thread run Python bytecode at a time, so the eight threads take turns on one core's worth of work — concurrency without parallelism. Use multiprocessing (separate processes, each with its own interpreter and its own GIL), or, more often, push the work into an engine such as DuckDB, Polars or Spark, whose compiled code releases the GIL and uses every core. See Processes, threads & the GIL.

2. Three slow jobs: (a) 2,000 API calls that each spend 300 ms waiting for a response; (b) a per-record calculation written in pure Python; (c) a group-by over 80 GB of Parquet. For each, choose between threads, multiprocessing and a columnar engine, and say what decides it.

   Show answer

   (a) Threads — the work is waiting on the network, and the GIL doesn't stop threads from waiting concurrently. (b) Multiprocessing — it is CPU-bound Python, so only separate processes run truly in parallel. (c) A columnar engine — it uses all cores in compiled code and spills to disk when memory runs low. What decides it is where the time goes (waiting versus computing) and whether the computing is your Python or something an engine already does; when an engine can do the job, prefer it over clever Python parallelism. See the approaches table and CPU & parallelism.

3. A pandas script runs cleanly on a 100 MB sample. On the 40 GB production file the log ends with the single word Killed — no traceback, no Python exception. There is no bug in the code. What happened, why was there no error message, and what question should have been asked before loading?

   Show answer

   read_csv materializes the whole file in RAM, and the in-memory form is often 2–10× the size on disk, so the process asked for far more memory than the machine has. The operating system cannot grant it and kills the process outright (on Linux, the OOM killer) — the process vanishes rather than raising an exception, so Python never gets to print anything. The question is "how big is this in RAM?", not "how big is it on disk?". See Memory & out-of-memory.

4. A job has been running for an hour. In top its %CPU sits in single digits, I/O wait is high, the disk is busy, and RES is flat. Someone proposes moving it to a machine with four times the cores. Will that help? What would?

   Show answer

   No — the job is I/O-bound: it spends its time waiting for bytes while the CPU idles, so more cores would idle too. Speed it up by reading less (a columnar format, filters, only the needed columns), by overlapping the waits with concurrency, or with faster storage. More cores, or a vectorized engine's parallelism, is the cure for the opposite picture — CPU pinned near 100%. Naming which of the two you have comes first, so you don't optimize the wrong thing. See Disk & I/O.

5. A group-by that got pandas killed completes in DuckDB on the same laptop — but takes noticeably longer than DuckDB does when the data fits in memory. Explain both halves of that using the memory hierarchy.

   Show answer

   Data must be in RAM for the CPU to work on it, and RAM is the small, scarce tier. pandas tries to hold everything there at once and is killed; DuckDB streams through the data and spills what doesn't fit to disk, so it survives. But each step down the hierarchy is orders of magnitude slower — disk is roughly 1,000× slower than RAM — so work that touches spilled data pays that price. Slower-but-finishes beats fast-until-killed. See the memory hierarchy and Let the engine use your machine.

Prediction: No — it will fail with an out-of-memory error
(the process gets Killed).

Why: read_csv MATERIALIZES the whole file in RAM as a DataFrame.
The in-memory form is typically larger than the 50 GB on disk
(Python object + type overhead), so it needs well over 50 GB —
but the laptop only has 16 GB. The OS can't grant it and kills
the process. Disk size (50 GB) is fine; RAM (16 GB) is the wall.

Two fixes (any two):
  1. STREAM/CHUNK it: read_csv(..., chunksize=...) and process a
     piece at a time, never holding the whole file in RAM.
  2. PUSH IT DOWN to an engine that spills to disk:
       import duckdb
       duckdb.sql("SELECT region, sum(amount) FROM 'data.csv'
                   GROUP BY region").df()
     DuckDB scans the file without materializing it all and uses
     all your cores.
  3. (Bonus) Convert to Parquet first and read only needed columns,
     turning random work into fast sequential column scans.

Takeaway

• Many pipeline failures are a resource limit: out of memory, full disk, a saturated CPU, or one busy core while the rest idle. Read the failure that way before blaming the tool.
• RAM is fast and scarce, disk is about a thousand times slower, and a process that asks for more RAM than exists is killed by the OS with no warning.
• In-memory data is often several times larger than the file on disk, so ask how big it is in RAM, not on disk, before you load it.
• Stream or chunk the data, or push the work into an engine that spills to disk, instead of materializing the whole dataset.
• The GIL stops Python threads running CPU-bound code in parallel. Use processes, or better an engine like DuckDB, Polars or Spark, for CPU work, and threads for waiting on I/O.
• Decide first whether a job is I/O-bound or CPU-bound, because the fix differs: read less for I/O, add cores or a vectorized engine for CPU. top or htop shows which.

Real pipelines also pull data across a network, so next we look at what happens between machines. → Networking & APIs

Further reading

• Operating Systems: Three Easy Pieces — Arpaci-Dusseau and Arpaci-Dusseau. A free textbook on processes, virtual memory and I/O.
• The USE Method — Brendan Gregg. Checking utilization, saturation and errors for each resource in turn.
• top(1) — Linux man-pages project. What %CPU, RES and load average mean in the output.
• Global interpreter lock — Python documentation. The definition and what it limits.
• multiprocessing — Python documentation. Process-based parallelism that avoids the GIL.