API Gateway Pattern: The Front Door to Your Microservices

An API gateway is a server that acts as the single entry point for all client requests into a backend system. It sits in front of your services and handles routing, authentication, rate limiting, SSL termination, and request transformation — so each individual service does not have to.

In practice: a client sends one request to api.example.com. The gateway authenticates the token, checks the rate limit, transforms the request if needed, routes it to the right upstream service, and returns the response. The downstream service never sees unauthenticated or malformed traffic.

You have ten microservices. Every client — mobile app, web app, third-party partner — needs to talk to all of them. They all need authentication. They all need rate limiting. They all need logging.

You could add all of this to every service. Or you could put a single component in front of them all.

That's the API gateway.

Without gateway:
Mobile App → /users    → User Service    (auth + rate limiting here)
Mobile App → /orders   → Order Service   (auth + rate limiting here)
Mobile App → /products → Product Service (auth + rate limiting here)
 
With gateway:
Mobile App → API Gateway → /users    → User Service
                         → /orders   → Order Service
                         → /products → Product Service
(auth, rate limiting, logging done once, in the gateway)

The gateway handles cross-cutting concerns so your services don't have to.

API Gateway Core Functions

An API gateway is more than a router. Here are the six core functions it performs and why each matters.

Routing

The gateway maps incoming paths to upstream services. /api/users/* goes to the user service, /api/orders/* goes to the order service. Services can change their internal URL structure or be completely replaced without clients ever knowing — the gateway absorbs the change.

Path-based routing is the most common pattern, but gateways also support header-based routing (route mobile traffic to a mobile-optimised service), query-param routing, and weighted routing (send 10% of traffic to a new service version for canary deploys).

Authentication

Verifying tokens once at the gateway is far more efficient than duplicating auth logic across every service. The gateway validates a JWT or API key, extracts the user identity, and passes that identity downstream via headers (X-User-ID, X-User-Role). Services trust those headers — they never touch token verification.

This also means rotating your auth library or changing your JWT secret only requires a change in one place.

Rate Limiting

Enforce limits per client at the gateway: 100 requests per minute per IP, 10,000 per day per API key. The gateway tracks usage (often in Redis for distributed counters) and returns 429 Too Many Requests before the request even reaches your service.

Without gateway-level rate limiting, a single abusive client can overwhelm every service in your system simultaneously.

Load Balancing

Multiple instances of a service running for availability or scale — the gateway distributes traffic across them using round-robin, least-connections, or IP-hash algorithms. When an instance fails health checks, the gateway stops sending traffic to it automatically.

SSL Termination

Clients connect over HTTPS. The gateway decrypts the TLS connection, then forwards requests to services over plain HTTP inside your private network. This is faster (services avoid TLS overhead), simpler to manage (one cert in one place), and means services don't need their own TLS configuration.

Request/Response Transformation

The gateway can modify requests before forwarding and modify responses before returning:

• Strip internal headers clients should never see
• Add X-Request-ID headers for distributed tracing
• Translate REST responses to GraphQL format
• Aggregate responses from multiple services into one payload
• Convert XML to JSON for legacy service compatibility

API Gateway vs Reverse Proxy vs Load Balancer

These three components are often confused because they overlap in functionality. Here's how they differ:

nginx sits in an interesting position: it is a reverse proxy that can also do basic load balancing and basic rate limiting. Used as a gateway it works well for simple cases but lacks dynamic configuration, plugin ecosystems, and developer portal features.

HAProxy is optimised purely for load balancing at very high throughput — it does not do authentication or rate limiting.

Kong is a full API gateway built on top of nginx. It adds dynamic configuration, a plugin ecosystem (auth, rate limiting, logging, monitoring), and a REST Admin API — all without reloading.

AWS API Gateway is the managed cloud option. Zero infrastructure to run, deep AWS integration (Cognito, Lambda authorizers, WAF, CloudWatch), but you're locked into AWS and costs can get high at scale.

The practical rule: use a reverse proxy if you just need to forward traffic, add an API gateway when you need to control who accesses your APIs and how.

Popular API Gateway Solutions

Kong (Open Source + Enterprise)

Kong is the most widely deployed open-source API gateway. It runs on top of nginx and is extended via plugins (400+ available). Configuration is done via REST API or declarative YAML — no nginx reload needed when adding routes or changing auth config.

Best for: Self-hosted, on-premises, or Kubernetes-native deployments where you want full control. Kong Ingress Controller is the standard way to use Kong in Kubernetes.

Limitations: Operational complexity — you manage the Kong instances, the database (PostgreSQL or Cassandra for clustering), and the configuration.

AWS API Gateway

Fully managed by AWS. You define routes in the console or via CDK/Terraform, link them to Lambda functions or HTTP backends, and AWS handles scaling, TLS, and availability.

Best for: AWS-native architectures, Lambda-heavy backends, teams that want zero ops overhead.

Limitations: Vendor lock-in, cold start latency for Lambda integrations, cost scaling with requests ($3.50 per million HTTP API requests, higher for REST API).

Azure API Management (APIM)

Microsoft's managed gateway. Strong developer portal, extensive policy engine (XML-based policies for transformation and auth), and AD integration.

Best for: Enterprise teams on Azure, especially with existing Active Directory infrastructure.

Limitations: Complex pricing tiers, policy syntax is verbose XML.

nginx

The battle-tested reverse proxy. With lua-nginx-module or OpenResty, nginx can do most gateway functions. Without Lua, it handles routing, SSL termination, basic rate limiting.

Best for: Simple setups, teams that already know nginx, high-performance static routing without plugin overhead.

Limitations: Dynamic configuration requires reload, no built-in developer portal, auth requires custom Lua or external auth service.

Traefik

A cloud-native proxy that auto-discovers services from Docker labels or Kubernetes annotations. Configuration updates automatically as containers start and stop — no manual config file changes.

Best for: Docker Compose and Kubernetes environments where services change frequently.

Limitations: Less mature plugin ecosystem than Kong, enterprise features require Traefik Enterprise.

Comparison Table

Building a Simple API Gateway in Node.js

Understanding how an API gateway works mechanically helps you use and configure one better. Here is a minimal but complete gateway built with Express and http-proxy-middleware.

Setup

npm install express http-proxy-middleware express-rate-limit jsonwebtoken

Complete Gateway Implementation

// gateway.js
const express = require('express');
const { createProxyMiddleware } = require('http-proxy-middleware');
const rateLimit = require('express-rate-limit');
const jwt = require('jsonwebtoken');
 
const app = express();
 
// ─── Service Registry ────────────────────────────────────────────────────────
const services = {
  users:    { target: 'http://user-service:8001',    prefix: '/api/users' },
  orders:   { target: 'http://order-service:8002',   prefix: '/api/orders' },
  products: { target: 'http://product-service:8003', prefix: '/api/products' },
};
 
// ─── Rate Limiting ───────────────────────────────────────────────────────────
const limiter = rateLimit({
  windowMs: 60 * 1000,  // 1 minute
  max: 100,             // 100 requests per minute per IP
  standardHeaders: true,
  legacyHeaders: false,
  handler: (req, res) => {
    res.status(429).json({
      error: 'Too Many Requests',
      retryAfter: Math.ceil(req.rateLimit.resetTime / 1000),
    });
  },
});
 
app.use(limiter);
 
// ─── Auth Middleware ─────────────────────────────────────────────────────────
const PUBLIC_KEY = process.env.JWT_PUBLIC_KEY;
 
function authMiddleware(req, res, next) {
  // Skip auth for health checks
  if (req.path === '/health') return next();
 
  const authHeader = req.headers['authorization'];
  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing or malformed Authorization header' });
  }
 
  const token = authHeader.slice(7);
 
  try {
    const payload = jwt.verify(token, PUBLIC_KEY, { algorithms: ['RS256'] });
 
    // Pass user identity downstream via headers
    req.headers['x-user-id']   = String(payload.sub);
    req.headers['x-user-role'] = payload.role || 'user';
    req.headers['x-user-email'] = payload.email || '';
 
    // Remove the raw token before forwarding (services don't need it)
    delete req.headers['authorization'];
 
    next();
  } catch (err) {
    if (err.name === 'TokenExpiredError') {
      return res.status(401).json({ error: 'Token expired' });
    }
    return res.status(401).json({ error: 'Invalid token' });
  }
}
 
app.use(authMiddleware);
 
// ─── Request ID ──────────────────────────────────────────────────────────────
app.use((req, res, next) => {
  req.headers['x-request-id'] = req.headers['x-request-id']
    || `gw-${Date.now()}-${Math.random().toString(36).slice(2, 9)}`;
  next();
});
 
// ─── Logging ─────────────────────────────────────────────────────────────────
app.use((req, res, next) => {
  const start = Date.now();
  res.on('finish', () => {
    console.log(JSON.stringify({
      requestId: req.headers['x-request-id'],
      method:    req.method,
      path:      req.path,
      status:    res.statusCode,
      duration:  Date.now() - start,
      userId:    req.headers['x-user-id'],
    }));
  });
  next();
});
 
// ─── Routes ──────────────────────────────────────────────────────────────────
for (const [name, config] of Object.entries(services)) {
  app.use(
    config.prefix,
    createProxyMiddleware({
      target: config.target,
      changeOrigin: true,
      pathRewrite: { [`^${config.prefix}`]: '' },
      on: {
        error: (err, req, res) => {
          console.error(`Proxy error [${name}]:`, err.message);
          res.status(502).json({ error: 'Bad Gateway', service: name });
        },
      },
    })
  );
}
 
// ─── Health Check ────────────────────────────────────────────────────────────
app.get('/health', (req, res) => res.json({ status: 'ok' }));
 
// ─── 404 Handler ─────────────────────────────────────────────────────────────
app.use((req, res) => {
  res.status(404).json({ error: `No route for ${req.method} ${req.path}` });
});
 
app.listen(8080, () => console.log('Gateway running on :8080'));

This 100-line gateway handles routing, JWT authentication, rate limiting, request ID propagation, structured logging, and graceful proxy errors. A production gateway (Kong, AWS API Gateway) adds clustering, persistence, hot config reloads, and a management API on top of these fundamentals.

Testing the Gateway

# Get a token
TOKEN=$(curl -s -X POST http://auth-service/token \
  -H "Content-Type: application/json" \
  -d '{"username":"test","password":"pass"}' | jq -r .token)
 
# Call through the gateway
curl http://localhost:8080/api/users/123 \
  -H "Authorization: Bearer $TOKEN"
 
# Trigger rate limit (run 101 times)
for i in $(seq 1 101); do
  curl -s -o /dev/null -w "%{http_code}\n" \
    http://localhost:8080/api/users/123 \
    -H "Authorization: Bearer $TOKEN"
done

API Gateway in Microservices Architecture

The Backend for Frontend (BFF) Pattern

The BFF pattern takes the API gateway idea further: instead of one generic gateway for all clients, you build a dedicated gateway per client type.

Mobile App     → Mobile BFF     → [User Service, Notification Service]
Web App        → Web BFF        → [User Service, Order Service, Product Service]
Partner API    → Partner BFF    → [Order Service, Inventory Service] (restricted)
Internal Tools → Internal BFF   → [All Services]

Why separate gateways per client?

• Mobile apps need smaller payloads (limited bandwidth), different fields, and different pagination than web apps
• Partners need access to a restricted subset of endpoints with different rate limits
• Web apps can aggregate multiple service calls into one response — mobile can't afford the latency of doing this client-side
• Each BFF can be owned by the team that owns the client — the mobile team controls the mobile BFF

SoundCloud coined the BFF term. Netflix uses it extensively — their device-specific BFFs aggregate data from 30+ microservices into a single response tailored for each device type (TV, phone, browser). A TV app needs a different data shape than a mobile app even for the same conceptual resource.

How Netflix Uses API Gateways

Netflix operates at a scale where gateway design is a major engineering challenge. Their Zuul gateway (open-sourced) handles hundreds of billions of requests per day. Key design decisions:

Dynamic routing: Routes are loaded from a persistence store and can be updated without restarting the gateway. This allows gradual rollout and instant rollback.

Resilience at the edge: Zuul implements circuit breakers per upstream service. If the recommendations service starts failing, Zuul returns cached or default responses — the failure does not cascade to the homepage.

Request passport: Each request gets a signed context object ("passport") injected at the gateway, containing user identity, device info, and request metadata. Services read the passport instead of re-fetching user data.

Canary routing: New service versions receive a percentage of traffic at the gateway level — no infrastructure changes needed.

Gateway in a Kubernetes Cluster

In Kubernetes, the gateway is typically implemented as an Ingress controller (nginx Ingress, Kong Ingress Controller, Traefik) or using the newer Gateway API spec.

# Kubernetes Gateway API example
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api-routes
spec:
  parentRefs:
    - name: main-gateway
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api/users
      backendRefs:
        - name: user-service
          port: 8001
          weight: 90
        - name: user-service-v2   # Canary: 10% to new version
          port: 8001
          weight: 10
    - matches:
        - path:
            type: PathPrefix
            value: /api/orders
      backendRefs:
        - name: order-service
          port: 8002

API Gateway Performance

Latency Overhead

Adding any component to the request path adds latency. For an API gateway, realistic numbers:

For most APIs where service response time is 20ms–200ms, gateway overhead is negligible. For sub-10ms internal services where you're chaining many calls, measure before assuming a gateway is fine.

Caching at the Gateway

The gateway can cache upstream responses and return cached data for identical requests — reducing load on services and eliminating downstream latency entirely.

# nginx proxy cache
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=api_cache:10m max_size=1g inactive=60m;
 
server {
    location /api/products/ {
        proxy_cache api_cache;
        proxy_cache_valid 200 5m;        # Cache 200 responses for 5 minutes
        proxy_cache_valid 404 1m;
        proxy_cache_use_stale error timeout; # Serve stale on upstream error
        proxy_cache_key "$scheme$request_method$host$request_uri$http_authorization";
        add_header X-Cache-Status $upstream_cache_status;
    }
}

Cache key design matters: include the Authorization header (or the extracted user ID) if responses differ per user. For public, user-agnostic data (product catalog, pricing), omit user context for maximum cache hit rate.

Connection Pooling

A gateway maintains a pool of persistent connections to upstream services. Without pooling, each inbound request would open a new TCP connection to the upstream — expensive at scale.

With connection pooling, the gateway reuses existing connections. For 10,000 requests per second, the difference between pooled and unpooled connections can be 30–50ms per request and significant CPU load on upstream services.

# nginx upstream with connection pooling
upstream user_service {
    server user-service-1:8001;
    server user-service-2:8001;
    keepalive 32;           # Keep 32 idle connections per worker
    keepalive_timeout 60s;  # Hold idle connections for 60 seconds
}

Simple Example: nginx as Gateway

For small setups, nginx does basic gateway work:

server {
    listen 443 ssl;
    server_name api.example.com;
 
    ssl_certificate     /etc/ssl/certs/api.crt;
    ssl_certificate_key /etc/ssl/private/api.key;
 
    # Rate limiting
    limit_req_zone $binary_remote_addr zone=api:10m rate=100r/m;
 
    # Route to user service
    location /api/users/ {
        limit_req zone=api burst=20 nodelay;
        proxy_pass http://user-service:8001/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Request-ID $request_id;
    }
 
    # Route to order service
    location /api/orders/ {
        limit_req zone=api burst=20 nodelay;
        proxy_pass http://order-service:8002/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Request-ID $request_id;
    }
}

Good for: simple routing, rate limiting, SSL termination. Not great for: complex auth logic, dynamic configuration without reloading.

Kong: Production API Gateway

Kong is a popular open-source gateway built on nginx. Configure it via API or declarative YAML — no code changes needed.

# kong.yml — declarative configuration
_format_version: "3.0"
 
services:
  - name: user-service
    url: http://user-service:8001
    routes:
      - name: users-route
        paths:
          - /api/users
 
  - name: order-service
    url: http://order-service:8002
    routes:
      - name: orders-route
        paths:
          - /api/orders
 
plugins:
  - name: jwt
    config:
      secret_is_base64: false
 
  - name: rate-limiting
    config:
      minute: 100
      hour: 1000
      policy: redis
 
  - name: request-id
    config:
      header_name: X-Request-ID
 
  - name: response-transformer
    config:
      remove:
        headers:
          - X-Internal-Service-Version
          - X-Pod-Name

Kong plugins handle auth, rate limiting, logging, and response transformation — all configured, not coded.

AWS API Gateway

If you're on AWS, API Gateway integrates with Lambda, ECS, and other services:

{
  "openapi": "3.0.1",
  "paths": {
    "/users/{userId}": {
      "get": {
        "x-amazon-apigateway-integration": {
          "type": "HTTP_PROXY",
          "uri": "http://user-service.internal/users/{userId}",
          "httpMethod": "GET",
          "passthroughBehavior": "when_no_match"
        }
      }
    }
  }
}

AWS API Gateway gives you: built-in auth (Cognito, Lambda authorizers), usage plans per API key, CloudWatch logging, and WAF integration — no infrastructure to manage. Good when you're already deep in AWS and want zero ops overhead.

Traefik: Gateway for Containers

Traefik reads Docker or Kubernetes labels and configures itself automatically:

# docker-compose.yml
services:
  api-gateway:
    image: traefik:v3.3
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.websecure.address=:443"
      - "--certificatesresolvers.letsencrypt.acme.email=ops@example.com"
    ports:
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
 
  user-service:
    image: user-service:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.users.rule=PathPrefix(`/api/users`)"
      - "traefik.http.routers.users.entrypoints=websecure"
      - "traefik.http.routers.users.middlewares=auth@file,ratelimit@file"
 
  order-service:
    image: order-service:latest
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.orders.rule=PathPrefix(`/api/orders`)"
      - "traefik.http.routers.orders.entrypoints=websecure"

Traefik auto-discovers services as containers start and stop. Ideal for Kubernetes.

When Not to Use a Gateway

Small monolith: One service with a few endpoints does not need a gateway. Put auth middleware in your app.

Added latency: Every request goes through an extra hop. For latency-sensitive internal APIs (sub-5ms targets), measure whether the overhead is acceptable.

Single point of failure: If the gateway goes down, everything goes down. You need redundancy — multiple gateway instances plus a load balancer in front. A poorly configured gateway is worse than no gateway.

Not a substitute for service security: Services behind the gateway should still validate inputs. Do not assume internal traffic is safe. Defense in depth means services enforce their own authorization even after the gateway authenticates the user.

Gateway vs Service Mesh

These solve different problems:

API gateway: North-south traffic (clients to services). One entry point for auth and rate limiting.

Service mesh (Istio, Linkerd): East-west traffic (service to service). Handles mTLS between services, circuit breaking, and retries for internal calls.

In production microservices systems, you often use both: gateway for client traffic, service mesh for internal service communication.

FAQ

What is an API gateway and do I need one?

An API gateway is a server that sits in front of your backend services and handles routing, authentication, rate limiting, and request transformation for all incoming client traffic. You need one when you have multiple services that share cross-cutting concerns (auth, rate limiting, logging) — without a gateway, you duplicate that logic in every service. For a single monolithic service, a gateway adds unnecessary complexity. For two or more services with external clients, a gateway pays for itself quickly.

What is the difference between an API gateway and a reverse proxy?

A reverse proxy forwards requests to backend servers and is primarily about traffic forwarding and SSL termination. An API gateway is a superset: it does everything a reverse proxy does, plus authentication, rate limiting per API key/user, request/response transformation, a developer portal, and usage analytics. nginx is a reverse proxy; Kong (built on nginx) is an API gateway. The line blurs because most gateways use a reverse proxy as their core engine.

Should I use Kong, AWS API Gateway, or nginx?

Use nginx if you need simple routing and SSL and already know nginx well — it handles the basics with minimal overhead. Use Kong if you need full gateway features (auth plugins, rate limiting, dynamic config, developer portal) and want to self-host. Use AWS API Gateway if you are already on AWS, your backend is Lambda-heavy, and you want zero infrastructure management. The decision usually comes down to: are you on AWS and want managed ops (AWS API Gateway), or do you want control and portability (Kong)?

How does an API gateway handle authentication?

The gateway validates the credential on every request — typically a JWT verified against a public key, or an API key looked up in a database. On success, it extracts the user identity and injects it into the forwarded request as headers (X-User-ID, X-User-Role). Downstream services trust those headers and never touch token verification themselves. This means auth logic lives in one place: if you rotate keys or change JWT libraries, you update one component, not every service.

What is the BFF (Backend for Frontend) pattern?

BFF (Backend for Frontend) is a pattern where you build a dedicated API gateway per client type rather than one generic gateway for all clients. A mobile BFF returns compact payloads optimised for small screens and limited bandwidth. A web BFF can aggregate multiple service calls into one response because browsers can handle the complexity. A partner BFF exposes only the endpoints partners are allowed to use. Each BFF is owned by the team that owns that client surface. Netflix uses per-device BFFs to aggregate data from 30+ microservices into a single response tailored for TVs, phones, and browsers.

Can an API gateway be a single point of failure?

Yes, and it's the most common objection to the gateway pattern. If the gateway goes down, no client can reach any service. The mitigation is standard: run multiple gateway instances, put a load balancer (or cloud NLB) in front of them, and deploy across availability zones. Most managed gateways (AWS API Gateway, Azure APIM) handle this for you. For self-hosted gateways (Kong, nginx), you own the redundancy. A well-deployed gateway is more reliable than the services behind it — the risk is a badly deployed one.

How do I implement rate limiting in an API gateway?

Rate limiting in a gateway works by tracking a counter per identifier (IP address, API key, user ID) within a time window, then rejecting requests that exceed the limit. In a single instance this is an in-memory counter. In a distributed gateway (multiple instances), you need a shared store — Redis is the standard choice. The response on limit hit is 429 Too Many Requests with a Retry-After header. Common strategies: fixed window (simple, can burst at window boundary), sliding window (smoother), and token bucket (allows short bursts). Kong's rate-limiting plugin and AWS API Gateway usage plans both support Redis-backed distributed counters out of the box.

Key Takeaways

• An API gateway is the single entry point for all clients — it routes to the right service and handles cross-cutting concerns once
• Core functions: routing, authentication, rate limiting, SSL termination, load balancing, request/response transformation
• nginx for simple setups, Kong for feature-rich self-hosted, AWS API Gateway for AWS-native, Traefik for containers
• Pass user identity from gateway to services via headers — services trust the gateway, never re-verify tokens
• The BFF pattern extends this: one dedicated gateway per client type (mobile, web, partner)
• Gateway latency overhead is 1–15ms depending on the tool and plugins — measure if sub-10ms matters
• You need redundancy — multiple gateway instances plus a load balancer in front; a gateway is a potential single point of failure
• Do not replace service-level security with gateway security alone

An API gateway removes boilerplate from every service. The complexity does not disappear — it consolidates into one place you control.

Related reading: Rate Limiting Your API · REST API Design Best Practices · Circuit Breaker Pattern