A Worked Design: Snip at Scale

Requirements and scope

A design review starts by pinning down what the system must do and how much, because those numbers decide every later choice. Snip is a URL shortener with click analytics. The functional requirements are small: create a short code for a long URL, redirect a code to its URL, and count clicks. Edits and deletes exist; custom codes are a feature a user can request. Everything else — accounts, billing, a web UI — is out of scope for the design that has to scale.

The non-functional requirements are where the work is. A redirect must be fast, because a person is waiting for the destination page: target a p99 under 50 ms of server time (chapter 08). Links must be durable — losing a user's short link is losing their data, so it must survive a machine failure. Redirects must stay available even when analytics or the dashboard is down. Counts may be approximate and a little behind; nobody audits a click total to the unit.

The scale is the one Start Here fixed: 100 million redirects a day (about 1,200 per second average, 6,000–12,000 at peak), 1 billion stored links, and 1 million new links a day. That read-heavy, write-light, count-everything shape is the whole reason the design looks the way it does.

The estimate

Before any boxes, the back-of-the-envelope pass from chapter 01 tells us which resource runs out first.

TRAFFIC   reads  100M/day  = ~1,200 /s avg, 6,000-12,000 /s peak
          writes 1M links/day = ~12 /s   (creates are a rounding error)
          clicks 1 per redirect = ~1,200 /s avg, ~12,000 /s peak

STORAGE   links   1B x 500 B        = 500 GB  (+ ~180 GB/year)
          clicks  100M/day x 100 B  = 10 GB/day = ~3.6 TB/year

CACHE     hot 1% of links (10M) take ~99% of reads
          hot set 10M x 500 B       = 5 GB  -> fits one Redis node
          at 99% hit, DB sees ~12 /s avg, ~120 /s peak

BANDWIDTH ~500 B per response x 12,000 /s = ~6 MB/s = ~48 Mbit/s peak

Read it off: bandwidth is trivial. The link table at 500 GB fits on one server for size, but 1 billion rows and a growing click log do not, and the click write rate — up to 12,000 a second — is what a single primary feels first. The hot set is 5 GB, so a cache turns 1,200 reads a second into roughly a dozen at the database. The design follows from those four facts: cache hard, keep the redirect path short, separate the click writes, and shard the store.

API, codes, and schema

Two endpoints carry the traffic:

POST /api/links   {url, custom_code?}  -> 201 {code, short_url}
GET  /{code}                            -> 302 Location: <original url>

A short code is 7 characters of base62 ([0-9A-Za-z]), which is 62⁷ ≈ 3.5 trillion values — a thousand times the billion links we expect, so random codes almost never collide. Generating a code randomly and inserting it with a uniqueness constraint (retry on the rare clash) avoids a central counter and leaks nothing about volume, unlike sequential IDs that let anyone enumerate every link by counting up. A custom code is the one case that needs a real uniqueness check: two users must not both claim /launch, which is the single place a stronger guarantee earns its cost (chapter 05).

The schema is deliberately plain. The links table is the source of truth, sharded by hash(code) (chapter 04). A second table keyed by owner answers "all my links" as a single-shard query, a deliberate denormalization. Click counts live in their own counter store, fed asynchronously; raw events go to a log and then an analytics store.

-- sharded by hash(code)
links(code PK, long_url, owner_id, created_at, status)
-- denormalized for the dashboard, sharded by owner_id
links_by_owner(owner_id, code, title, created_at)
-- counters, updated by the click consumers (eventual)
link_counts(code PK, clicks)

The read path

A redirect is the request we optimize for everything, because it is 99% of the traffic and the one with a human waiting. The path is kept deliberately short — no fan-out, so the tail stays close to the tail of its parts (chapter 08):

Every stage is a lever from an earlier chapter. The app servers hold no session or local state, so the load balancer can send a request to any of them and we add replicas as traffic grows (chapter 02). The redirect returns a 302, not a 301, so browsers keep asking us and we keep seeing clicks. Redis absorbs about 99% of lookups; the 1% that miss do a single-shard point read on a replica, a few milliseconds when healthy. At the 12,000-per-second peak that leaves only about 120 reads a second reaching the shards — light enough that they run with plenty of headroom.

The write paths

There are two writes, and they could not be more different. Creating a link is rare — about 12 a second — and must be durable and immediately usable by its creator. It writes to the shard's primary, and for a short window after, that user's reads go to the primary so they never get a "not found" from a lagging replica: read-your-writes, bought with a routing rule rather than a protocol (chapter 05).

Recording a click is the hard write: up to 12,000 a second, bursty, and on the hot path of the request we just promised to keep fast. So it does not touch the database synchronously. The redirect handler appends a small click event to a durable log and returns the 302 without waiting; consumers aggregate the events into counters off to the side (chapter 06):

The log absorbs the burst the database could not (chapter 08). Consumers batch a few thousand events and issue one update per code, so a viral link becomes a single clicks = clicks + N instead of ten thousand updates fighting over one row. Events carry an id so a redelivery under at-least-once counts once. The counts are eventual and approximate, which the requirements allowed. The raw events roll up into a columnar analytics store — the data-side problem, covered in Data Systems Design — chosen because its access pattern is scans over billions of rows (chapter 07).

Consistency and failure

With copies in a cache, replicas, and shards, the design has to say what each read may see and what happens when a part dies. Snip spends strong consistency in exactly one place and degrades everywhere else.

Failure follows one rule: the redirect is the product; everything else is optional (chapter 09). If the click pipeline is down, redirects do not wait — events are buffered in the log, and if even that backs up they are sampled or dropped. If a shard's primary dies, a replica is promoted behind fencing; reads keep serving from cache and the surviving replicas, and only new links on that shard pause for the tens of seconds it takes to fail over. If Redis restarts empty, cache-aside falls through to the database — correct but slower, and dangerous, because the shards sized for ~120 reads a second would suddenly meet thousands. So the fall-through is guarded with per-key request coalescing and load shedding, and the cache is re-warmed rather than left to a stampede (chapter 03).

The scaling ladder

You do not build the diagram above on day one. You reach the end of it by removing one bottleneck at a time, which is the loop Start Here promised. The same product grows through these rungs:

Most products never climb past rung 3 or 4, and that is the point: you spend the next lever's complexity only when a measured bottleneck forces it, never because the diagram has more boxes. Each rung bought headroom against the resource that ran out, and exposed the next one.

Defending it

In a review — or an interview — the design is only as good as the trade-offs you can name. The ones this design would be challenged on:

• Why shard by hash(code) and not by owner? Because the hot query is the redirect, keyed by code; sharding by code makes it a single-shard point lookup. The dashboard query ("my links") is served by the owner-keyed copy instead. You optimize the shard key for the 99% path and denormalize for the rest.
• Why eventual counts? Because exact, synchronous counts would put 12,000 contended writes a second on the lookup path, trading the redirect's latency for a precision nobody needs. The requirement said approximate was fine; the design cashes that in.
• What breaks first under 10× growth? The click pipeline's consumers and the analytics store, not the redirect path — the log absorbs the burst, but aggregation and storage grow with total volume. That is where the next capacity work goes.
• Single region? This design is one region. Going multi-region adds cross-region replication lag and the hard question of where writes are accepted — a real jump in consistency complexity (chapter 05), taken only when latency for distant users or regional failure justifies it.

Notice that every answer is a sentence about a bottleneck and the price of the move that cleared it. That is the whole method of this guide, applied until the design is done.

✓ Check yourself

Answer from memory before opening each one.

1. Snip returns a 302 for redirects, not a 301, even though a 301 would let browsers and the CDN cache the redirect harder and cut load. Why is that the right call here?

   Show answer

   A 301 is permanent: browsers cache it indefinitely, so an edited or deleted link would keep sending old visitors to the old URL, and — just as important for Snip — repeat visits would never reach the server, so clicks would go uncounted. The 302 keeps every visit coming back to us, which the analytics product depends on. The small extra load is the price of correctness and counting.

2. A reviewer says: "clicks are writes at up to 12,000 a second, and links are only 500 GB, so the database is write-bound, not size-bound — why shard the links at all?" How do you answer?

   Show answer

   They are right that the click writes are the heavy write load, which is exactly why those do not go to the links database at all — they go to a log and are aggregated into a separate counter store. The links database is left with ~12 creates a second and cached reads. Links are sharded for size and row count (1 billion rows, growing) and to survive node loss with per-shard replicas, not for the click write rate. Separating the two write paths is what lets each be sized for its real load.

3. You are at rung 3 (one database, a Redis cache, no replicas) and a load test shows the redirect p99 is fine at the 99% hit rate but spikes badly whenever Redis is restarted for maintenance. What is happening, and which rung do you climb, and how do you make the restart safe in the meantime?

   Show answer

   An empty cache sends the full read load — thousands a second — straight at the one database, a stampede the single primary cannot absorb. Climb to rung 4 (read replicas) so reads have somewhere to go, and add request coalescing so concurrent misses for the same key hit the DB once. In the meantime, re-warm Redis before taking traffic, or fail over to a replica cache, rather than restarting cold under load.

4. Product asks for a live, exact, to-the-click counter shown on each link's page, updating in real time. What do you tell them, and what would you offer instead?

   Show answer

   Exact, synchronous, per-click updates would put every click as a contended write on one row — a hot-key problem that would also drag the redirect path if it shared the store. You offer near-real-time instead: counters updated every second or few from the aggregators, which looks live to a human and keeps the write path cheap. If a specific view truly needs the exact number, compute it on demand from the raw event store, not on the hot path.

5. A teammate proposes jumping straight to the rung-7 design — sharded, multi-tier, async, circuit-breakered — for a brand-new product expecting a few thousand links a day. What is the argument against it?

   Show answer

   At that load a single box with a cache handles everything with room to spare, and the complex design costs real money and operational burden (routing tiers, consumers, a consensus store, more failure modes) to remove bottlenecks that do not exist yet. Each rung should be climbed when a measured bottleneck forces it. Building the top of the ladder first is paying the whole cost up front for headroom you may never use, and carrying the complexity slows every change in the meantime.

Takeaway

• Start a design by fixing the numbers; Snip's read-heavy, write-light, count-everything shape dictates every later choice.
• Keep the hot path short and cached: a redirect is CDN, load balancer, stateless app, Redis (~99%), and at most one single-shard read — no fan-out, so the tail stays small.
• Separate the two writes. Link creates are rare and durable with read-your-writes; clicks are a high-volume stream that goes to a log and is aggregated off the hot path, eventual and approximate.
• Spend strong consistency in one place — the shard map — and let redirects and counts be eventual. The redirect is the product; everything else may degrade.
• You reach the full design one rung at a time: split tiers, cache, replicate, go async, shard, harden — each move forced by a measured bottleneck, never by the shape of the diagram.
• In a review, defend each choice as "this bottleneck, this move, this price." A design you can defend that way is a design that scales for reasons, not by hope.

That is the whole guide in one system. Go back to the chapter list to revisit any lever, or Start Here for the mental model that ties them together.

Further reading

• Designing Data-Intensive Applications — Martin Kleppmann, O'Reilly, 2017. The single best companion to this guide; every lever here is treated in depth there.
• Site Reliability Engineering — Google, O'Reilly, 2016. How these designs are actually run: SLOs, overload, and cascading failures, free online.
• The System Design Primer — an open collection of system-design patterns and worked examples; good for seeing the same levers applied to other products.