Software Engineering · Deep dive

Designing for Scale

How a backend service goes from one box to millions of requests a day without falling over. The laws that bound every system, the levers in the order you reach for them, and the trade-off hiding behind each one — worked through a single running example: Snip, a URL shortener with click analytics.

Scaling is not a feature you add; it is a sequence of bottlenecks you remove. Every system has one resource it runs out of first — a CPU, a disk, a lock, a single database — and the craft is finding that resource, then either making it bigger (scale up) or spreading the work across more of them (scale out). Do that, and a new bottleneck appears behind it. The whole discipline is this loop, run with judgement about which move is worth its cost.

This guide builds the toolkit from first principles. It assumes you can write a service and query a database, but not that you have run one under real load. Each chapter adds one lever — caching, replication, sharding, queues — explains the mechanism and the price, and shows where it applies to Snip. The last chapter puts every lever together on one whiteboard.

Section A · Orient

00 Start Here The one mental model — find the bottleneck, then move it or split it — the running example, and three reading paths from beginner to interview-ready.

Section B · The building blocks

01 The Numbers and the Laws Latency versus throughput, scaling up versus out, Little's Law, the latency numbers every engineer should know by heart, and the back-of-envelope math that sizes a system in two minutes. 02 Load Balancing & Statelessness Why in-process state is the enemy of horizontal scale, how to push it out, load-balancing algorithms, L4 versus L7, health checks, and when session affinity helps versus hurts. 03 Caching The cache hierarchy from browser to CDN to application to database; cache-aside, write-through and write-back; the hard part — invalidation and TTLs; and the thundering-herd stampede. 04 Scaling the Database Read replicas and replication lag, connection pooling, the limits of a single primary, then partitioning and sharding — choosing a shard key and living with hot partitions and cross-shard queries.

Section C · Consistency, flow & failure

05 Consistency & Its Price CAP and PACELC, the consistency models from linearizable to eventual, quorums, consensus (Raft) in a page, and where strong consistency earns its latency cost versus where eventual is fine. 06 Asynchronous Work & Queues Decoupling with queues and event streams, backpressure, at-least-once versus exactly-once delivery, idempotency keys, dead-letter queues, and the transactional outbox. 07 Choosing a Data Store by Shape Relational, key-value, document, wide-column, search, time-series, blob and graph — matching the access pattern to the store, and the real cost of polyglot persistence. 08 Tail Latency & Capacity Why the average lies, p99 and p999, how fan-out amplifies the tail, head-of-line blocking, hedged requests, load testing, and planning capacity with real headroom. 09 Reliability & Graceful Degradation Redundancy and failover, timeouts and retries with backoff and jitter, circuit breakers, bulkheads, rate limiting and load shedding — how a system degrades instead of collapsing.

Section D · Put it together

10 A Worked Design: Snip at Scale Take Snip from a single box to millions of redirects a day, reaching for every lever in order — the estimates, the schema, the caching and sharding choices, and what you would defend in a design review.

How to read this guide

Chapters build in order, each adding one lever to the toolkit, so a first read front to back pays off. If you are short on time, pick a path:

New to backend scale

Read 00 → 04 in order; that is the core — the laws, statelessness, caching, and the database. Then read 08 (tail latency) and 09 (reliability). Come back for 05–07 when you hit them.

Engineer brushing up

Skim 01, read 03, 04 and 05 closely (caching, sharding, consistency), then 06 and 08. Chapter 10 ties it together.

System-design interview soon

00, 01 (the numbers), 04, 05, 08, then 10 as a dry run. 09 for the reliability questions interviewers add at the end.