Build Beyond Backend
system design

System Design Interview: Design a URL Shortener

Walk through designing a URL shortener like bit.ly from scratch. Covers hashing, database design, caching, scalability, and analytics with practical decisions.

By Akash Sharma·6 min read
#system design
#system design interview
#url shortener
#scalability
#redis
#database design
#backend

URL shortener is one of the most common system design interview questions. It looks simple but has interesting depth: hashing, redirects, scale, analytics.

Let's design it end to end.

Requirements

Functional:

  • Shorten a URL: POST /shorten → returns short code like xyz123
  • Redirect: GET /xyz123 → 301/302 redirect to original URL
  • Optional: custom aliases (/my-custom-slug), expiry, click analytics

Non-functional (scale):

  • 100M URLs created per day
  • 10:1 read/write ratio → 1B redirects per day
  • Low latency redirects (< 50ms)
  • Data retention: 5 years

Let's turn that into numbers:

  • Writes: 100M / 86,400s ≈ 1,200 writes/sec
  • Reads: 1.2B / 86,400s ≈ 14,000 reads/sec
  • Storage: 100M URLs/day × 500 bytes × 365 days × 5 years ≈ 90 TB

Core Design: Generating Short Codes

The key question: how do you turn a long URL into a short unique code?

Option 1: Hash the URL

python
import hashlib
import base64
 
def shorten(url: str) -> str:
    hash_bytes = hashlib.md5(url.encode()).digest()
    # Take first 6 characters of base64-encoded hash
    return base64.urlsafe_b64encode(hash_bytes)[:6].decode()
 
shorten("https://example.com/very/long/url")  # → "abc123"

Problem: Hash collisions. Two different URLs could map to the same code. You'd need to detect collisions and append a counter.

Bigger problem: Same URL always gives the same code. If two users shorten the same URL, they share analytics. Usually not desirable.

Option 2: Random Code + DB Check

python
import secrets
import string
 
CHARS = string.ascii_letters + string.digits  # 62 characters
 
def generate_code(length: int = 7) -> str:
    return ''.join(secrets.choice(CHARS) for _ in range(length))
 
def shorten(url: str) -> str:
    while True:
        code = generate_code()
        if not db.exists(f"url:{code}"):  # Check for collision
            db.set(f"url:{code}", url)
            return code

62^7 = 3.5 trillion combinations. Collision probability is negligible until you have billions of URLs.

Problem: DB check per generation. For truly parallel writes, two workers could generate the same code simultaneously and both pass the check.

Option 3: Auto-Increment ID + Base62 Encode (Best)

Generate a unique auto-incrementing ID, then convert it to base62. No collision possible.

python
def encode_base62(num: int) -> str:
    chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
    if num == 0:
        return chars[0]
    
    result = []
    while num:
        result.append(chars[num % 62])
        num //= 62
    return ''.join(reversed(result))
 
def decode_base62(code: str) -> int:
    chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
    result = 0
    for char in code:
        result = result * 62 + chars.index(char)
    return result
 
encode_base62(1)          # → "b"
encode_base62(1000000)    # → "4c92"
encode_base62(10000000)   # → "FXsk"

ID 1 → b, ID 2 → c, ..., growing as you add URLs. 7-character base62 = 3.5 trillion URLs.

Sequence generation: Use PostgreSQL SERIAL or a Redis counter (atomic increment):

python
# Redis: atomic counter across all servers
def get_next_id() -> int:
    return redis.incr("url_counter")
 
def shorten(url: str, user_id: str) -> str:
    url_id = get_next_id()
    code = encode_base62(url_id)
    
    db.execute(
        "INSERT INTO urls (id, code, original_url, user_id, created_at) VALUES (%s, %s, %s, %s, NOW())",
        url_id, code, url, user_id
    )
    
    return f"https://short.ly/{code}"

Database Schema

sql
CREATE TABLE urls (
    id          BIGSERIAL PRIMARY KEY,
    code        VARCHAR(10) UNIQUE NOT NULL,
    original_url TEXT NOT NULL,
    user_id     BIGINT,
    created_at  TIMESTAMP DEFAULT NOW(),
    expires_at  TIMESTAMP,           -- NULL = never expires
    is_active   BOOLEAN DEFAULT TRUE
);
 
CREATE INDEX idx_urls_code ON urls(code);  -- Fast lookup by code
CREATE INDEX idx_urls_user ON urls(user_id, created_at);  -- User's URLs
 
CREATE TABLE clicks (
    id          BIGSERIAL PRIMARY KEY,
    url_id      BIGINT REFERENCES urls(id),
    clicked_at  TIMESTAMP DEFAULT NOW(),
    ip_address  INET,
    user_agent  TEXT,
    referer     TEXT,
    country     CHAR(2)
);
 
CREATE INDEX idx_clicks_url ON clicks(url_id, clicked_at);

The Redirect Endpoint

python
from fastapi import FastAPI, HTTPException
from fastapi.responses import RedirectResponse
import redis
 
app = FastAPI()
r = redis.Redis()
 
@app.get("/{code}")
async def redirect(code: str):
    # 1. Check cache first
    original_url = r.get(f"url:{code}")
    
    if not original_url:
        # 2. Cache miss — check DB
        row = db.fetchone("SELECT original_url, expires_at, is_active FROM urls WHERE code = %s", code)
        
        if not row or not row.is_active:
            raise HTTPException(status_code=404)
        
        if row.expires_at and row.expires_at < datetime.utcnow():
            raise HTTPException(status_code=410, detail="URL expired")
        
        original_url = row.original_url
        # Cache with TTL
        r.setex(f"url:{code}", 3600, original_url)
    
    # 3. Record click asynchronously (don't slow down redirect)
    background_tasks.add_task(record_click, code)
    
    # 4. Redirect
    return RedirectResponse(url=original_url, status_code=301)
    # 301 = permanent (browser caches) → fastest repeat visits
    # 302 = temporary (browser re-requests each time) → accurate analytics

301 vs 302: Use 301 for performance (browser caches, repeat visits skip your server). Use 302 for accurate click counting (every redirect goes through your server). Most URL shorteners use 302 for analytics.

Caching Strategy

Redirects are pure reads. Cache everything.

python
# Cache hot URLs in Redis
# Key: "url:{code}" → original URL
# TTL: 1 hour (or longer for viral links)
 
# Cache hit rate will be very high — power law distribution
# Top 20% of URLs = 80% of traffic
# Those will be hot in cache

At 14,000 reads/sec with high cache hit rate, your DB only handles cache misses — maybe 1,000-2,000 queries/sec. One PostgreSQL server handles this easily.

Analytics: Counting Clicks at Scale

Recording a click to the DB on every redirect is expensive at scale (14K writes/sec). Use a write buffer:

python
import asyncio
 
click_buffer = []
 
async def record_click(code: str, metadata: dict):
    click_buffer.append({"code": code, "ts": time.time(), **metadata})
 
async def flush_clicks():
    while True:
        await asyncio.sleep(5)  # Flush every 5 seconds
        if click_buffer:
            batch = click_buffer.copy()
            click_buffer.clear()
            db.executemany("INSERT INTO clicks ...", batch)

Or use a message queue (Kafka) — redirect service publishes click events, analytics service consumes and stores them separately from the critical path.

Handling Scale

At 100M URLs/day and 1B redirects/day, one server isn't enough.

Read scaling: Redirect endpoint is stateless. Add more servers behind a load balancer. Redis cache is shared.

Write scaling: URL creation (1,200/sec) is easily handled by one PostgreSQL server. If not, shard by user ID.

Redis: Single Redis handles ~100K ops/sec. Redis Cluster for more.

Global distribution: CDN at the edge? Not quite — redirects are dynamic (lookup by code). Use GeoDNS to route users to the nearest data center with local Redis + DB replica.

Complete Architecture

plaintext
                    ┌─── URL Service (write) ─── PostgreSQL (primary)
Client ─── LB ──── │
                    └─── Redirect Service (read) ─── Redis Cache
                                                   └── PostgreSQL (replica)
                                                   └── Analytics Queue (Kafka)
                                                          └── Analytics DB

Key Takeaways

  • Auto-increment ID + base62 encoding is the cleanest approach — no collision possible
  • Cache redirects in Redis with long TTL — most traffic hits cache, not DB
  • Use 302 for click analytics, 301 for performance (browser caches)
  • Record clicks asynchronously — don't make the redirect wait for a DB write
  • Separate read and write paths — redirect service (high read scale) vs creation service
  • 7-character base62 = 3.5 trillion unique codes — more than enough for any scale

URL shortener is a great exercise: simple on the surface, but it teaches hashing, caching, read/write separation, and async processing.

Related reading: Database Indexing Explained · Redis Caching Explained · Caching Strategies Explained

Enjoyed this article?

Get weekly insights on backend architecture, system design, and Go programming.

Related Posts

Continue reading with these related posts

database
6 min

Database Replication Explained: Primary, Replica, and Failover

Learn how database replication works. Covers primary-replica setup, synchronous vs asynchronous replication, read replicas, and automatic failover strategies.

Read more
system design
6 min

Caching Strategies Explained: Cache-Aside, Write-Through, and More

Learn the four main caching strategies: cache-aside, read-through, write-through, and write-behind. When to use each pattern with Redis examples.

Read more
system design
6 min

Microservices vs Monolith: Which Should You Build?

Learn the real tradeoffs between microservices and monolithic architecture. When microservices make sense, when they don't, and how to migrate without pain.

Read more