Rate Limiting: How It Works and How to Protect Your API

Rate limiting is a technique that controls how many requests a client can make to your API within a defined time window. When a client exceeds that limit, the API rejects further requests — typically with an HTTP 429 Too Many Requests response — until the window resets.

It protects APIs by ensuring no single client (malicious or otherwise) can monopolize server resources, cause performance degradation for other users, or rack up costs on expensive operations.

Without rate limiting, one user sending 10,000 requests per minute can slow your server to a crawl, cause legitimate users to experience timeouts, and spike your infrastructure costs in minutes.

Why APIs Need Rate Limiting

Rate limiting solves several real problems simultaneously:

• Availability under abuse: A flood of requests from a single source cannot bring down your service for everyone else
• Fair resource distribution: Heavy users cannot degrade the experience for lighter users
• Cost control: Expensive operations like AI inference, image processing, or third-party API calls can be capped to prevent runaway bills
• Business tier enforcement: Free users get 100 calls/day, paid users get 10,000 — rate limits make this enforceable
• Accidental self-inflicted damage: Bugs in a client's retry logic can accidentally hammer your API; rate limits cap the blast radius

Rate Limiting Algorithms Compared

Different algorithms make different trade-offs between accuracy, memory usage, and burst handling. Here is how each one works with code and a comparison at the end.

Fixed Window Counter

Divide time into fixed windows (e.g., each minute). Count requests in the current window. Reject when count exceeds the limit.

import redis
import time
 
r = redis.Redis(host='localhost', port=6379)
 
def fixed_window(client_id: str, limit: int = 100, window_seconds: int = 60) -> bool:
    """Returns True if the request is allowed."""
    window = int(time.time() / window_seconds)
    key = f"fw:{client_id}:{window}"
    count = r.incr(key)
    if count == 1:
        r.expire(key, window_seconds)
    return count <= limit

The boundary problem: A client can send 100 requests at 12:00:59 and 100 more at 12:01:00 — 200 requests in 2 seconds, yet both windows show exactly 100. This double-spend at window boundaries is the core weakness.

Pros: Extremely simple, O(1) memory per client, fast. Cons: Boundary burst allows 2x the intended limit.

Leaky Bucket

Requests enter a queue (the bucket) and are processed at a fixed output rate. If the bucket is full, new requests are dropped.

import time
import threading
from collections import deque
 
class LeakyBucket:
    def __init__(self, rate: float, capacity: int):
        self.rate = rate          # requests per second to process
        self.capacity = capacity  # max queue size
        self.queue = deque()
        self.lock = threading.Lock()
 
    def allow(self) -> bool:
        with self.lock:
            now = time.time()
            # Drain processed requests
            while self.queue and self.queue[0] + (1.0 / self.rate) <= now:
                self.queue.popleft()
            if len(self.queue) < self.capacity:
                self.queue.append(now)
                return True
            return False

Pros: Smooth, predictable output rate — good for rate-limiting outgoing calls to third-party APIs. Cons: Bursting is not allowed; legitimate burst traffic gets queued or dropped. Poor fit for most inbound API use cases.

Token Bucket

A bucket fills with tokens at a steady rate up to a maximum capacity. Each request consumes one token. If the bucket is empty, the request is rejected.

• Bucket capacity: 100 tokens (maximum burst)
• Refill rate: 10 tokens per second
• A client can burst to 100 requests instantly, then is limited to 10 per second

import redis
import time
 
class TokenBucketRateLimiter:
    def __init__(self, redis_client, capacity: int, refill_rate: float):
        self.r = redis_client
        self.capacity = capacity        # Max tokens (burst limit)
        self.refill_rate = refill_rate  # Tokens added per second
 
    def is_allowed(self, client_id: str) -> bool:
        now = time.time()
        key = f"tb:{client_id}"
 
        # Atomic Lua script prevents race conditions
        lua_script = """
        local tokens_key = KEYS[1]
        local last_key = KEYS[2]
        local capacity = tonumber(ARGV[1])
        local refill_rate = tonumber(ARGV[2])
        local now = tonumber(ARGV[3])
 
        local tokens = tonumber(redis.call('GET', tokens_key) or capacity)
        local last_refill = tonumber(redis.call('GET', last_key) or now)
 
        local elapsed = now - last_refill
        tokens = math.min(capacity, tokens + elapsed * refill_rate)
 
        if tokens >= 1 then
            tokens = tokens - 1
            redis.call('SET', tokens_key, tokens, 'EX', 3600)
            redis.call('SET', last_key, now, 'EX', 3600)
            return 1
        else
            return 0
        end
        """
 
        result = self.r.eval(
            lua_script, 2,
            f"{key}:tokens", f"{key}:last",
            self.capacity, self.refill_rate, now
        )
        return bool(result)

Pros: Allows legitimate burst traffic, enforces average rate, flexible. Cons: Two Redis keys per client; requires Lua for atomicity.

Sliding Window Log

Store the timestamp of every request. On each new request, count timestamps within the last N seconds. Reject if count exceeds limit.

import redis
import time
 
r = redis.Redis(host='localhost', port=6379)
 
def sliding_window_log(client_id: str, limit: int = 100, window_seconds: int = 60) -> bool:
    now = time.time()
    key = f"swl:{client_id}"
    window_start = now - window_seconds
 
    pipe = r.pipeline()
    # Remove timestamps outside the window
    pipe.zremrangebyscore(key, 0, window_start)
    # Count remaining
    pipe.zcard(key)
    # Add current request timestamp
    pipe.zadd(key, {str(now): now})
    pipe.expire(key, window_seconds)
    results = pipe.execute()
 
    count = results[1]
    return count < limit

Pros: Perfectly accurate — no boundary burst problem. Cons: Stores one entry per request; memory grows linearly with traffic. A client making 10,000 requests per minute needs 10,000 sorted set entries.

Sliding Window Counter

A practical middle ground. Weight the previous window's count by the fraction of the current window already elapsed.

import redis
import time
import math
 
r = redis.Redis(host='localhost', port=6379)
 
def sliding_window_counter(client_id: str, limit: int = 100, window_seconds: int = 60) -> bool:
    now = time.time()
    current_window = math.floor(now / window_seconds)
    prev_window = current_window - 1
 
    current_key = f"swc:{client_id}:{current_window}"
    prev_key = f"swc:{client_id}:{prev_window}"
 
    pipe = r.pipeline()
    pipe.get(current_key)
    pipe.get(prev_key)
    results = pipe.execute()
 
    current_count = int(results[0] or 0)
    prev_count = int(results[1] or 0)
 
    # How far into the current window are we?
    elapsed_fraction = (now % window_seconds) / window_seconds
    # Weight the previous window inversely
    estimated = prev_count * (1 - elapsed_fraction) + current_count
 
    if estimated >= limit:
        return False
 
    pipe = r.pipeline()
    pipe.incr(current_key)
    pipe.expire(current_key, window_seconds * 2)
    pipe.execute()
    return True

Example: 80 requests in the previous window, 40 in the current one, 30 seconds elapsed out of 60 seconds:

estimated = 80 × (1 - 0.5) + 40 = 80

The client is considered to have used 80 of their 100 requests.

Pros: Near-accurate, O(1) memory per client, fast. Cons: Slightly approximate (not exact), but error is always bounded.

Algorithm Comparison Table

Default recommendation: Sliding Window Counter for most APIs. Token Bucket when burst tolerance is a product requirement.

Implementing Rate Limiting with Redis

Redis is the standard tool for distributed rate limiting because it is fast, supports atomic operations, and is already deployed in most backend stacks. Here is a complete, production-ready implementation using sliding window counter:

import redis
import time
import math
from typing import Tuple
from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import JSONResponse
 
r = redis.Redis(host='localhost', port=6379, decode_responses=True)
app = FastAPI()
 
 
def check_rate_limit(
    client_id: str,
    limit: int = 100,
    window_seconds: int = 60
) -> Tuple[bool, int, int, int]:
    """
    Sliding window counter rate limiter.
 
    Returns:
        (allowed, limit, remaining, reset_at)
    """
    now = time.time()
    current_window = math.floor(now / window_seconds)
    prev_window = current_window - 1
    elapsed_fraction = (now % window_seconds) / window_seconds
 
    current_key = f"rl:{client_id}:{current_window}"
    prev_key = f"rl:{client_id}:{prev_window}"
 
    pipe = r.pipeline()
    pipe.get(current_key)
    pipe.get(prev_key)
    results = pipe.execute()
 
    current_count = int(results[0] or 0)
    prev_count = int(results[1] or 0)
 
    estimated = int(prev_count * (1 - elapsed_fraction) + current_count)
    reset_at = int((current_window + 1) * window_seconds)
    remaining = max(0, limit - estimated - 1)
 
    if estimated >= limit:
        return False, limit, 0, reset_at
 
    # Increment current window
    pipe = r.pipeline()
    pipe.incr(current_key)
    pipe.expire(current_key, window_seconds * 2)
    pipe.execute()
 
    return True, limit, remaining, reset_at
 
 
@app.middleware("http")
async def rate_limit_middleware(request: Request, call_next):
    # Rate limit by IP for unauthenticated requests
    client_ip = request.client.host
    api_key = request.headers.get("X-API-Key")
 
    # Use API key if present, fall back to IP
    client_id = f"key:{api_key}" if api_key else f"ip:{client_ip}"
 
    # Different limits per authentication level
    limit = 1000 if api_key else 60
 
    allowed, limit_val, remaining, reset_at = check_rate_limit(
        client_id, limit=limit, window_seconds=60
    )
 
    if not allowed:
        return JSONResponse(
            status_code=429,
            content={"error": "Rate limit exceeded", "retry_after": reset_at - int(time.time())},
            headers={
                "X-RateLimit-Limit": str(limit_val),
                "X-RateLimit-Remaining": "0",
                "X-RateLimit-Reset": str(reset_at),
                "Retry-After": str(reset_at - int(time.time())),
            }
        )
 
    response = await call_next(request)
    response.headers["X-RateLimit-Limit"] = str(limit_val)
    response.headers["X-RateLimit-Remaining"] = str(remaining)
    response.headers["X-RateLimit-Reset"] = str(reset_at)
    return response

The key design decisions here:

• Middleware applies limits before any route handler runs
• API key holders get higher limits than anonymous IP-based clients
• Rate limit headers are always returned, not just on rejection
• The Retry-After value tells clients exactly how long to wait

Rate Limiting in Distributed Systems

The Problem with Multiple Servers

A single server with an in-memory counter works fine. Add a second server and the counter is wrong. If you have two servers each allowing 100 requests per minute, a client can actually make 200 requests per minute by alternating between them.

Client → Load Balancer → Server A (counter: 95/100) ✓
Client → Load Balancer → Server B (counter: 95/100) ✓
# Total: 190 requests allowed — almost double the limit

Solution: Centralized Redis Counter

All servers share a single Redis instance as the counter store. The rate limiter code runs on each server, but reads and writes the same Redis keys.

Client → Load Balancer → Server A ─┐
                                    ├── Redis (single source of truth)
Client → Load Balancer → Server B ─┘

This is why every production rate limiter uses Redis or a similar centralized store. The implementation above already does this — check_rate_limit always reads from and writes to the shared Redis instance.

Redis Cluster Considerations

For very high traffic, a single Redis node becomes a bottleneck. Two approaches:

Option 1: Redis Cluster with key sharding Rate limit keys shard across Redis nodes by client ID. Each client's counter lives on one node. This scales horizontally but requires careful key design to avoid hot spots.

Option 2: Local + global two-layer limiting Each server maintains a local counter (in memory) for fast rejection of obvious abusers. A global Redis counter enforces the true limit. Local counters reset more frequently and act as a pre-filter.

# Two-layer: fast local pre-check, then Redis
local_counters = {}  # In-memory, per-server
 
def two_layer_check(client_id: str, local_limit: int = 20, global_limit: int = 100) -> bool:
    # Fast local check (resets every 10 seconds)
    local_key = f"{client_id}:{int(time.time() / 10)}"
    local_counters[local_key] = local_counters.get(local_key, 0) + 1
    if local_counters[local_key] > local_limit:
        return False  # Reject without hitting Redis
 
    # Global Redis check
    allowed, _, _, _ = check_rate_limit(client_id, limit=global_limit)
    return allowed

Sticky Sessions Approach

An alternative to shared Redis is sticky sessions — the load balancer always routes a given client to the same server. Each server maintains its own counter independently.

Downside: Sticky sessions undermine horizontal scaling and cause uneven load distribution. A single "sticky" server can become overloaded. Not recommended for rate limiting specifically.

Rate Limit Headers

Good APIs tell clients their rate limit status on every response, not just when they hit the limit. This allows clients to self-throttle before getting rejected.

Standard Headers

Why They Matter for API Consumers

Without rate limit headers, a client has two options when it gets a 429: retry immediately (usually wrong) or implement exponential backoff with no information about when to stop backing off.

With headers, a well-behaved client can:

1. Check X-RateLimit-Remaining before each request
2. Slow down proactively when remaining drops below a threshold
3. On 429, read Retry-After and wait exactly that long — no guessing

# Client-side rate limit awareness (example consumer code)
import time
import requests
 
class RateLimitAwareClient:
    def get(self, url: str) -> requests.Response:
        response = requests.get(url)
 
        remaining = int(response.headers.get('X-RateLimit-Remaining', 999))
        reset_at = int(response.headers.get('X-RateLimit-Reset', 0))
 
        if response.status_code == 429:
            retry_after = int(response.headers.get('Retry-After', 60))
            print(f"Rate limited. Waiting {retry_after}s...")
            time.sleep(retry_after)
            return self.get(url)  # Retry after waiting
 
        # Self-throttle when close to limit
        if remaining < 10:
            wait = max(0, reset_at - time.time())
            time.sleep(wait / remaining)  # Spread remaining requests over the window
 
        return response

The Retry-After header is defined in RFC 7231 and should always be included on 429 responses. X-RateLimit-* headers are not formally standardized but are universally understood.

Rate Limiting at Different Layers

Rate limiting can be applied at multiple points in your stack. Each layer has different trade-offs.

CDN Level (Cloudflare, Fastly, CloudFront)

Requests that violate rate limits are blocked before they reach your infrastructure. Zero server cost for rejected requests.

# Cloudflare Workers rate limit example
addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})
 
async function handleRequest(request) {
  const ip = request.headers.get('CF-Connecting-IP')
  const { success } = await RATE_LIMITER.limit({ key: ip })
 
  if (!success) {
    return new Response('Rate limit exceeded', {
      status: 429,
      headers: { 'Retry-After': '60' }
    })
  }
 
  return fetch(request)
}

Best for: Anonymous traffic, bot blocking, DDoS mitigation. Cannot differentiate by user identity or API key.

API Gateway Level (Kong, AWS API Gateway, Nginx)

Applied before requests reach application servers. Configuration-based, no code required.

Nginx rate limiting:

# nginx.conf
http {
    limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
 
    server {
        location /api/ {
            limit_req zone=api burst=20 nodelay;
            limit_req_status 429;
            proxy_pass http://backend;
        }
    }
}

Kong plugin configuration:

# Kong rate limiting plugin
plugins:
  - name: rate-limiting
    config:
      minute: 100
      hour: 1000
      policy: redis
      redis_host: redis
      redis_port: 6379

AWS API Gateway supports built-in throttling at the stage and method level — configured in the console or via CloudFormation, no code required.

Application Level (Your Code)

Most flexible. Can use business logic — different limits by user tier, endpoint cost, or custom rules. This is where the Redis implementations above live.

Which Layer Should You Use?

Use all of them in combination:

The layers complement each other. CDN stops volumetric attacks cheaply. API Gateway enforces service-level contracts. Application code enforces business rules.

DDoS vs Rate Limiting

Rate limiting and DDoS protection are related but solve different problems.

Rate limiting handles:

• A single misbehaving client (accidental or intentional)
• API scraping
• Runaway clients with retry bugs
• Enforcing usage tiers

Rate limiting does NOT handle:

• Distributed volumetric attacks from thousands of IPs (each IP stays under per-IP limits)
• Layer 3/4 network-level attacks (UDP floods, SYN floods) — these hit your network before any application code runs
• Slow-rate attacks designed to stay just under rate limit thresholds

DDoS protection (Cloudflare, AWS Shield, etc.) handles:

• Attacks from thousands of IPs simultaneously
• Network-layer attacks
• Protocol-level attacks
• Traffic pattern anomalies

Think of rate limiting as protection against one client misbehaving. DDoS protection handles coordinated attacks from many sources. You need both.

Rate limiting stops this:
  Client A → 10,000 req/min → BLOCKED after limit
 
Rate limiting does NOT stop this:
  Client A → 90 req/min  ─┐
  Client B → 90 req/min  ─┤→ 9,000 req/min combined → NOT BLOCKED per-client
  Client C → 90 req/min  ─┘
  ...100 clients...

The second scenario requires DDoS protection or global rate limiting (total requests across all clients), which requires different infrastructure.

FAQ

What is rate limiting and why do APIs need it?

Rate limiting controls how many requests a client can make to an API in a given time window. APIs need it to prevent any single client — whether a bug, a bot, or a bad actor — from consuming disproportionate server resources, causing degraded performance for other users, or incurring unexpected infrastructure costs.

What is the difference between rate limiting and throttling?

The terms are often used interchangeably, but there is a distinction: rate limiting rejects requests that exceed a threshold (returns 429). Throttling delays requests — queuing them and processing at a reduced rate. Rate limiting is more common for inbound API traffic. Throttling is more common when controlling outbound calls to third-party APIs where you want to stay under their limits without failing.

Which rate limiting algorithm should I use?

For most production APIs: Sliding Window Counter. It is accurate, uses constant memory (O(1) per client), and handles burst traffic reasonably. Use Token Bucket when your product explicitly needs to allow burst traffic (e.g., a user can fire 50 requests instantly but is limited to 10/second on average). Avoid Sliding Window Log at scale — the memory cost grows linearly with request count.

How do I implement rate limiting with Redis?

Use Redis INCR + EXPIRE for fixed window, or a sorted set (ZADD/ZREMRANGEBYSCORE) for sliding window log. For sliding window counter, maintain two keys — current window and previous window — and compute a weighted estimate. Use Lua scripts for atomic multi-step operations to prevent race conditions. The complete implementation is in the "Implementing Rate Limiting with Redis" section above.

How do I rate limit by IP vs by user?

Use the request's IP address (request.client.host) as the key for unauthenticated endpoints. For authenticated endpoints, use the user ID or API key as the key — this is more accurate because multiple legitimate users can share an IP (NAT, corporate networks). In production, combine both: per-IP limits for unauthenticated traffic, per-user limits for authenticated traffic, and a global limit as a ceiling for all traffic combined.

What HTTP headers should I return for rate limits?

Always return X-RateLimit-Limit (the maximum), X-RateLimit-Remaining (how many are left), and X-RateLimit-Reset (Unix timestamp when the window resets). On 429 responses, also return Retry-After (seconds until retry is safe). Return these headers on every response — not just 429 — so well-behaved clients can self-throttle before hitting the limit.

How does rate limiting work across multiple servers?

In-memory rate limiting breaks with multiple servers — each server has its own counter, so a client can make N × limit requests by hitting all N servers. The solution is a centralized counter store, almost always Redis. All servers write to the same Redis keys, so the counter is accurate regardless of which server handles each request. For very high traffic, use Redis Cluster with key sharding by client ID.

Key Takeaways

• Rate limiting caps requests per client per time window — it returns 429 when the limit is exceeded
• Sliding Window Counter is the right default: accurate, O(1) memory, handles burst
• Redis is the standard counter store because it is fast and supports atomic operations
• In distributed systems, all servers must share a single Redis counter — otherwise per-server counters are exploitable
• Always return X-RateLimit-* headers so clients can self-throttle
• Apply limits at multiple layers: CDN for volume, API Gateway for service contracts, application for business rules
• Rate limiting protects against individual bad actors; DDoS protection handles coordinated attacks from many IPs

Rate limiting is one of those things that seems unnecessary until you do not have it.

Related reading: REST API Design Best Practices · Redis Caching Explained