Networking & APIs

The client/server model

Strip away every buzzword and networking is two programs exchanging messages. One side asks; the other answers. The asker is the client (your script, your browser, your pipeline). The answerer is the server (an API, a database, a web service). A request goes out, a response comes back. That's the whole shape, and everything in this chapter is a detail hung on it.

A few things to notice now, because they shape everything later. The client always starts the conversation — the server can't push to you out of nowhere; it answers when asked. Each request is a round trip: out and back, with the network's delay paid both ways. And the same server happily talks to thousands of clients at once, which is exactly why a server you depend on might be slow or unavailable through no fault of yours.

The network stack, in plain language

For a request to find the right machine and arrive intact, several layers each do one job. You rarely touch them directly, but knowing what they do explains a lot of slowness and a lot of errors.

IP addresses & ports. Every machine on a network has an IP address — a number like 93.184.216.34 that says which machine. A port (like 443 or 5432) says which program on that machine, since one server runs many. Address + port together name an exact endpoint to talk to.

DNS — names to addresses. Humans can't remember IP numbers, so we use names like api.example.com. The Domain Name System is the network's phone book: before any request, your machine asks DNS to turn the name into an IP address. (This lookup is itself a tiny network round trip — and a surprisingly common source of "why is the first request slow?")

TCP vs UDP. Once you have an address and port, you need a way to move bytes. TCP is the reliable, ordered channel: it guarantees every byte arrives, in order, retransmitting anything lost — and almost everything you'll touch (HTTP, Postgres, Kafka) rides on it. UDP trades those guarantees for speed: it just fires packets and doesn't care if some are lost (think live video) — you'll meet it rarely.

HTTP up close

HTTP is the language clients and servers speak on the web — and the language nearly every API you'll pull from speaks. It's just structured text. A request has four parts, and so does a response. Here's a request:

GET /v1/orders?page=2 HTTP/1.1            ← method · URL/path · version
Host: api.example.com                     ← headers (metadata, key: value)
Authorization: Bearer abc123
Accept: application/json

(no body for a GET — a POST would put JSON here)

And the response that comes back:

HTTP/1.1 200 OK                           ← status line (code + reason)
Content-Type: application/json            ← headers
Content-Length: 1843

{ "orders": [ … ], "next_page": 3 }       ← body (the actual data)

The method states your intent — what you want done. There are a handful you'll use constantly:

The status code in the response tells you, at a glance, how it went. You don't need all of them — you need the families and a few specifics:

The single rule worth memorizing: 4xx is your problem, 5xx is theirs. Retrying a 4xx (you sent something wrong) just repeats the mistake; retrying a 5xx or 429 (their side) often succeeds once they recover. We'll lean on that in the reliability section.

Headers are the metadata around the message: who you are (Authorization), what format you'll accept (Accept), what format the body is (Content-Type), and often rate-limit counters telling you how many requests you have left. They're easy to overlook and frequently hold the clue when something behaves oddly.

REST APIs — your most common external source

Most third-party data you'll ever pull — payment records, weather, CRM contacts, GitHub stats — arrives through a REST API: a set of HTTP endpoints, each a URL representing a kind of resource, that you GET to read and usually get back as JSON. A vendor publishes one, hands you a key, and your pipeline becomes a client politely asking for pages of data. Two patterns dominate, and you must handle both.

Pagination. No sane API returns a million rows in one response. Instead it returns a page — say 100 records — plus a hint for getting the next one (a next_page number, a cursor token, or a Link header). You loop: fetch a page, process it, ask for the next, and stop when the hint runs out. Forgetting to paginate is the classic rookie bug — you grab page 1 and quietly miss 99% of the data.

Rate limits. APIs cap how fast you may call them (e.g. 60 requests/minute). Exceed it and you get a 429 Too Many Requests, often with a Retry-After header saying how long to wait. The right response is to slow down and back off — not to hammer harder. We'll do exactly that below.

Let's pull real data. This calls a free public API that needs no key, parses the JSON, and walks a few pages.

import requests

# A public, no-auth API. Each "post" is a small JSON object.
resp = requests.get(
    "https://jsonplaceholder.typicode.com/posts",
    params={"_page": 1, "_limit": 5},   # ask for page 1, 5 items
    timeout=10,                          # never wait forever (see below)
)

resp.raise_for_status()                  # turn a 4xx/5xx into an error
data = resp.json()                       # parse the JSON body into Python

print("status:", resp.status_code)
print("items on this page:", len(data))
print("first title:", data[0]["title"][:40])

Authentication & where secrets go

Most real APIs need to know who you are before they hand over data. That proof travels in a header (or sometimes a query parameter), and it comes in a few common flavors:

The mechanics matter less than the discipline around the secret itself. A key or token is a password — anyone who has it can act as you, run up your bill, or read your data. So the iron rule:

import os
import requests

# Set this in your shell first, e.g.:  export API_TOKEN="abc123"
token = os.environ["API_TOKEN"]          # read at runtime, never written in code

resp = requests.get(
    "https://api.example.com/v1/orders",
    headers={"Authorization": f"Bearer {token}"},
    timeout=10,
)

The token now lives in your shell or your deployment's secret store, not in the file you push to GitHub. Same code runs everywhere; each environment supplies its own value. That separation — config and secrets outside the code — is a habit that will keep you out of real trouble.

When the network fails — and it will

Here is the truth that separates a script from a pipeline: the network always fails eventually. A request hangs, a server returns 503, a connection drops mid-stream, DNS hiccups. Not if — when, and at scale, often. Robust code treats failure as normal and plans for it with three tools.

Timeouts. Never wait forever. A request with no timeout can hang indefinitely if the server stops responding, freezing your whole pipeline. Always set one (you saw timeout=10 above). A request that takes too long is itself a failure — treat it as one and move on.

Retries with exponential backoff. Many failures are transient — a blip, a momentary overload. The cure is to try again, but not instantly and not forever: wait a moment, retry, and if it fails again wait longer (1s, then 2s, then 4s — doubling). This exponential backoff gives a struggling server room to recover instead of a thundering herd of clients hammering it the instant it wobbles. Cap the number of attempts so you don't loop forever, and only retry the failures worth retrying — 5xx and 429, not 4xx.

import time
import requests

def get_with_retry(url, max_attempts=4, **kwargs):
    """GET a URL, retrying transient failures with exponential backoff."""
    for attempt in range(max_attempts):
        try:
            resp = requests.get(url, timeout=10, **kwargs)
            if resp.status_code < 500 and resp.status_code != 429:
                return resp          # success OR a 4xx — retrying won't help
        except requests.exceptions.RequestException:
            pass                     # connection error / timeout: fall through to retry

        if attempt < max_attempts - 1:
            wait = 2 ** attempt      # 1s, 2s, 4s, …  (exponential backoff)
            time.sleep(wait)

    raise RuntimeError(f"{url} failed after {max_attempts} attempts")

✓ Check yourself

Answer from memory before opening each one.

1. An extractor fetches 200,000 records one per request, sequentially, at 40 ms per round trip. How long does it spend purely waiting? The team's proposed fix is a link with ten times the bandwidth — how much does that help, and why?

   Show answer

   200,000 × 40 ms = 8,000 seconds, a little over two hours. More bandwidth barely helps: bandwidth is how much fits through the pipe per second, latency is how long one round trip takes, and they are independent — tiny responses don't fill the pipe, the job is just paying the round-trip delay 200,000 times. The fix is fewer round trips: larger pages or batches, or fetching in parallel. See Latency vs bandwidth.

2. A nightly pull from a vendor with about 12,000 customers lands exactly 100 rows every night. Every response is 200 OK and nothing is logged as an error. What is the most likely bug, and what should the extraction loop look like?

   Show answer

   The job reads page 1 and stops. APIs return a page — here 100 records — plus a hint for the next one (a next_page number, a cursor, or a Link header), and a successful status only means that page worked. Loop: fetch a page, process it, follow the hint, and stop only when the hint runs out or a page comes back empty. See REST APIs.

3. During one run your extractor receives a 400, a 401, a 429 and a 503. Which should be retried automatically, which should not, and what is the reasoning behind the split?

   Show answer

   Retry the 429 and the 503, with backoff; don't retry the 400 or 401. 4xx means you sent something wrong — a malformed request, a bad or missing token — so repeating it just repeats the mistake until you fix the request or the credentials. 5xx means their side broke or is overloaded, which often clears. 429 is the 4xx exception: you are being rate-limited, so slow down, honour Retry-After if it is sent, and try again later. See HTTP up close.

4. A teammate pastes an API token into extract.py, commits, notices ten minutes later, deletes the line and commits again. "Fixed — it's gone." Why is that wrong, and where should the token have lived?

   Show answer

   The token is still in git history, which keeps every committed version forever; on a public repo, scrapers harvest keys within minutes. Deleting the line changes only the latest snapshot, so the token has to be treated as leaked — and since a token is a password, anyone holding it can act as you until it is revoked and replaced. It belongs in an environment variable or a secrets manager, read at runtime (os.environ["API_TOKEN"]), with any file that holds it listed in .gitignore. See Authentication & where secrets go.

5. You adapt get_with_retry to POST "create order". One request times out on the client, the retry succeeds — and the customer now has two orders. Reconstruct what happened, and give the two ways to make retrying this call safe.

   Show answer

   The first request reached the server and created the order; only the response was lost or late, so the client saw a failure and sent it again. A timeout tells you that you didn't hear back, not that nothing happened. Either auto-retry only operations that are idempotent — doing them twice has the same effect as once, as with GET — or make the write idempotent by sending a unique request ID that the server de-duplicates on. See Why retried requests must be idempotent.

6. To cut the wait in question 1 you can request bigger pages or run many requests in parallel. The API allows 60 requests per minute. Which lever do you pull first, what decides it, and when does the other one win?

   Show answer

   Bigger pages first. With a rate limit, the number of requests is the scarce thing: parallel calls only hit the cap sooner and earn 429s, while a larger page moves more rows per allowed request and removes round trips outright. Parallelism wins when the limit is generous (or absent) but the page size is capped by the vendor, so the only way left to overlap the waiting is to have several requests in flight — still with backoff when the server pushes back. The vendor's docs tell you both numbers; read them first. See Rate limits and fewer round trips.

import time
import requests

def get_once_with_retry(url, params):
    """One GET, with a timeout and a single retry on a transient failure."""
    for attempt in range(2):                 # original try + 1 retry
        try:
            resp = requests.get(url, params=params, timeout=10)
        except (requests.exceptions.ConnectionError, requests.exceptions.Timeout):
            resp = None                      # network blip: fall through to the retry
        if resp is not None and resp.status_code < 500 and resp.status_code != 429:
            resp.raise_for_status()          # a real 4xx is our bug: raise now, never retry
            return resp.json()
        if attempt == 0:
            time.sleep(1)                    # brief backoff before the retry
    raise RuntimeError(f"failed to fetch {url} {params}")

def fetch_all_pages(url, page_size=20):
    items, page = [], 1
    while True:
        batch = get_once_with_retry(url, {"_page": page, "_limit": page_size})
        if not batch:                        # empty page → we're done
            break
        items.extend(batch)
        page += 1
    return items

all_posts = fetch_all_pages("https://jsonplaceholder.typicode.com/posts")
print("total items fetched:", len(all_posts))   # -> 100

Takeaway

• A request is a round trip, so latency, not bandwidth, usually dominates a pipeline that makes many small calls.
• A name resolves through DNS to an address and port, TCP gives a reliable ordered stream, TLS encrypts it, and HTTP is the layer you write.
• A 4xx status is your request's fault and retrying repeats the mistake. A 5xx or a 429 is the server's side and often succeeds on retry.
• Pull from a REST API by paginating, honoring rate limits and Retry-After, and reading the vendor docs for page size and auth.
• Treat keys and tokens as passwords: read them from the environment or a secret store, never hard-code them, and treat a committed secret as leaked.
• Set a timeout on every request and retry transient failures with exponential backoff, but only for requests that are safe to repeat.

One machine can now fetch data reliably. Next, what changes when one machine is not enough. → Distributed Systems I: Scaling Out

Further reading

• RFC 9110: HTTP Semantics — IETF, 2022. The definition of methods, status codes, headers and idempotent methods.
• RFC 6585: Additional HTTP Status Codes — IETF, 2012. Defines 429 Too Many Requests and its Retry-After use.
• RFC 6749: The OAuth 2.0 Authorization Framework — IETF, 2012. The token exchange behind the OAuth row in the auth table.
• RFC 9293: Transmission Control Protocol — IETF, 2022. How TCP delivers an ordered, reliable byte stream.
• Exponential Backoff And Jitter — AWS Architecture Blog, 2015. Why backoff needs jitter when many clients retry at once.
• Requests documentation — Python Requests project. Timeouts, sessions, raise_for_status and JSON handling.