CAP Theorem Explained: Consistency vs Availability in Distributed Systems

The CAP theorem states that any distributed data store can only guarantee two of three properties simultaneously: Consistency, Availability, and Partition Tolerance. Because network partitions are inevitable in any real distributed system, the practical choice always comes down to one question: when a partition occurs, do you prioritize correct data or system uptime?

This guide explains the CAP theorem from first principles, covers real database decisions (ZooKeeper, Cassandra, HBase, DynamoDB, PostgreSQL), introduces the PACELC extension, and helps you make the right CP vs AP choice for each part of your system.

The Three Properties of CAP

Consistency (C): Every read receives the most recent write — or an error. All nodes in the cluster return the same data at the same time. If you write "price = $100" to one node, any node you query next will return $100. There is no version lag between nodes.

Availability (A): Every request gets a non-error response. The system keeps responding even if some nodes are down. No timeouts, no "503 Service Unavailable." The response may not reflect the absolute latest write, but you always get something.

Partition Tolerance (P): The system continues operating even when network messages between nodes are lost or delayed. Nodes cannot always communicate — cables break, switches fail, cloud regions lose connectivity. Partition tolerance means the system keeps working anyway.

Why You Cannot Have All Three

Network partitions are not edge cases. They are a routine part of operating distributed systems at scale. Hardware fails. DNS times out. A misconfigured firewall drops packets between availability zones. In any real distributed system, you must tolerate partitions — the alternative is a system that becomes completely non-functional the moment any network hiccup occurs.

This means partition tolerance is mandatory. The real choice is between C and A during a partition event.

Consider this scenario: your database has two nodes in different data centers. A network partition splits them — they cannot communicate.

A write arrives at Node 1: "Set user balance to $500."

Node 1 cannot sync this to Node 2. Now what?

Option 1 — Prioritize Consistency (CP): Reject the write (or block all reads) until the partition resolves. Return an error. Data is always correct, but the system is unavailable during the partition.

Option 2 — Prioritize Availability (AP): Accept the write on Node 1 and serve reads from both nodes. Node 2 still returns the old value ($450). The system is up, but two clients reading from different nodes get different answers.

Neither option is universally better. The right choice depends on what your application can tolerate.

What a Network Partition Actually Looks Like

A network partition is any situation where nodes in your cluster cannot communicate reliably with each other. This includes:

• Full network split: Two data centers lose connectivity entirely for 30 seconds
• Partial partition: Some nodes can talk to each other but not all — creating a cluster that sees two different views of the world
• High latency masquerading as a partition: Packet loss so severe that timeouts fire before messages arrive
• Node failure: A node goes down — from the perspective of other nodes, this is indistinguishable from a partition

Partitions are more common than most engineers expect. In a 100-node cluster, even with 99.9% uptime per node, you will statistically see a node failure every day. Cloud providers experience intra-region connectivity issues multiple times per year. Cross-region links are even less reliable.

The question is not "will partitions happen?" — they will. The question is "what should my system do when they do?"

How Long Do Partitions Last?

Most network partitions resolve within seconds to a few minutes. But even a 30-second partition is enough to create divergent state across nodes if your system is under write load. Your CAP choice determines what users experience during those 30 seconds.

CAP Theorem in Practice: CP vs AP Systems

CP Systems: Correctness Over Uptime

CP databases refuse to serve inconsistent data. During a partition, they will return errors or block requests rather than risk serving stale or divergent values.

ZooKeeper (CP): ZooKeeper is the canonical CP system. It uses the ZAB (ZooKeeper Atomic Broadcast) consensus protocol. A write is only acknowledged after a majority quorum of nodes confirms it. During a partition that isolates the minority partition, those nodes stop serving writes and reads. This is by design — ZooKeeper is used for distributed coordination, leader election, and configuration management where correctness is non-negotiable. If ZooKeeper returned stale leader information, your entire cluster could elect two leaders simultaneously.

HBase (CP): HBase is built on top of HDFS and uses ZooKeeper internally for coordination. Writes go to a single RegionServer responsible for a given key range. If that RegionServer is unreachable, the write fails rather than going to an alternative node that might diverge. HBase optimizes for consistent reads at the cost of availability during failures.

PostgreSQL (CP): In single-node mode, PostgreSQL is trivially consistent (one node, no partition possible). With synchronous streaming replication, it becomes CP: a write is not acknowledged to the client until the standby confirms it received the WAL record. If the standby goes down, and you have synchronous_commit = on, writes block. Consistent data — but unavailable under partition.

etcd (CP): etcd uses the Raft consensus protocol. Like ZooKeeper, it requires a majority quorum to process writes. Used in Kubernetes for cluster state — if etcd returned inconsistent data about which pod is running where, Kubernetes would make catastrophically wrong scheduling decisions.

AP Systems: Uptime Over Correctness

AP databases stay operational during partitions, accepting that different nodes may temporarily hold different values. They rely on eventual consistency — given enough time without new writes, all nodes converge.

Cassandra (AP): Cassandra has no single leader. Every node is a peer. Writes go to multiple nodes simultaneously (determined by the replication factor). During a partition, Cassandra keeps accepting writes to reachable nodes. When the partition heals, Cassandra reconciles divergent values using last-write-wins (LWT) or custom merge logic. You can tune consistency per query — asking for agreement from ONE node (fast, potentially stale) or QUORUM nodes (slower, more consistent). But at its core, Cassandra chooses availability.

DynamoDB (AP by default): DynamoDB uses eventual consistency for reads by default. A write is acknowledged once it reaches enough nodes to satisfy the write concern, but reads may return data that hasn't caught up yet. You can opt into strongly consistent reads, but they cost twice as many read units and add latency. By default: AP.

CouchDB (AP): CouchDB's replication model is explicitly multi-master. Conflicts are allowed and surfaced to the application for resolution. This is the purest AP stance — availability is maximized, and the application is responsible for handling divergence.

Quick Reference: Where Each Database Falls

PACELC: The Better Model

CAP has a critical blind spot: it only describes behavior during a network partition. It says nothing about what happens during normal operation when everything is working fine.

This is where the PACELC theorem improves on CAP. Proposed by Daniel Abadi in 2012, PACELC states:

• If there is a Partition (P): the system must choose between Availability (A) and Consistency (C)
• Else (E) — when the system is running normally: the system must choose between Latency (L) and Consistency (C)

The key insight: even without a partition, you cannot have both low latency and strong consistency. To guarantee a write is consistent across all replicas, you must wait for all replicas to acknowledge it. Waiting adds latency. Every strongly consistent system pays a latency tax.

Why This Matters in Practice

Cassandra is not just AP (during partitions) — in PACELC terms, it is PA/EL: Partition-Available, Else-Low-latency. It sacrifices consistency both during partitions and during normal operation to maximize write throughput.

DynamoDB (default) is PA/EL: Available during partitions, low-latency else.

ZooKeeper is PC/EC: Partition-Consistent, Else-Consistent. During normal operation, ZooKeeper still enforces linearizability — and still pays the latency cost of that guarantee.

PostgreSQL with synchronous replication is PC/EC: consistent during partitions (blocks), and consistent during normal operation (waits for standby acknowledgment).

Spanner (Google's globally distributed SQL database) is an interesting case: PC/EC — it achieves strong consistency across continents using atomic clocks and GPS hardware. But it pays an extreme latency cost for cross-region writes (typically 100–200ms).

PACELC Classification of Common Systems

When to prefer EL (latency-optimized): User-facing writes where a 50ms latency difference is noticeable. High-throughput event ingestion. IoT data pipelines where the sheer volume makes synchronous replication impractical.

When to prefer EC (consistency-optimized): Any write where reading your own write immediately is required. Financial state. Inventory that cannot oversell. Distributed locks.

Consistency Models Beyond CAP

CAP uses the term "consistency" loosely — it really means linearizability. But the real world has a spectrum of consistency guarantees, and understanding them helps you reason about what your application can tolerate.

Strong Consistency (Linearizability)

Every operation appears to take effect instantaneously at some point between its invocation and completion. If Client A writes a value and then Client B reads it, Client B sees Client A's write — guaranteed, every time.

Cost: High. Requires coordination across all replicas before acknowledging a write. Adds latency proportional to the round-trip time between nodes.

When you need it: Bank balances. Inventory counts. Any value where reading the wrong answer causes financial or operational harm.

Sequential Consistency

All operations appear to execute in some total order, and the order seen by any individual client is consistent with their own operations. But two clients may see a different global ordering.

Cost: Medium. Easier to implement than linearizability because you don't need real-time ordering, just per-client consistency.

When it's enough: Collaborative editing where conflicts are resolved by server ordering. Message queues where per-producer ordering is required but cross-producer ordering is not.

Causal Consistency

If operation A causally precedes operation B (A's result was used to compute B's input), then all nodes must observe A before B. Operations with no causal relationship can be observed in any order.

Cost: Lower. Tracks causality metadata (vector clocks or similar) rather than requiring global coordination.

When it's enough: Social media threads where "you replied to my comment" must show the comment before the reply. Shopping cart operations where "add item then remove item" must be ordered. Most collaborative features that don't involve money.

Eventual Consistency

Given no new writes, all replicas will eventually converge to the same value. No guarantee on when. During that convergence window, different clients may see different values.

Cost: Lowest. Writes are acknowledged immediately. Replication happens asynchronously.

When it's enough: Product catalog (stale pricing for a second is acceptable). User profile photos. Social media engagement counts. Anything where "probably right soon" beats "definitely right eventually with downtime."

How These Models Map to Your Database Choices

• ZooKeeper, etcd, Spanner: Linearizable (strong)
• PostgreSQL (sync replication): Serializable (effectively strong)
• MongoDB (primary reads): Sequential (reads from primary are always current)
• Cassandra (QUORUM reads): Causal (with careful tuning)
• Cassandra (ONE reads), DynamoDB (default): Eventual

Designing for CAP: Real Decisions

CAP is not a choice you make once for your whole system. Different components of the same application have different consistency requirements. Here is how to think through it:

Payments and Financial Balances → CP

A user's account balance cannot be wrong. If you process a payment on Node 1 and a concurrent read on Node 2 returns a balance that doesn't reflect that payment, you may allow an overdraft. This is not acceptable.

Decision: PostgreSQL with synchronous replication, or a CP database. Accept the availability trade-off. If the database is unavailable, fail the transaction — do not guess.

Inventory / Stock Counts → CP (with caveats)

Overselling is a real business problem. If your warehouse has 1 unit of a product and two concurrent purchases see a count of 1, both will proceed — and one will fail at fulfillment.

Decision: CP for the authoritative inventory count. Use a database-level atomic decrement operation (Redis DECR, PostgreSQL UPDATE ... WHERE stock > 0 RETURNING *) to make the decrement and availability check atomic. Never read-then-write with eventual consistency for inventory.

Caveat: Some businesses accept a small oversell rate if it is cheaper to handle operationally than to run a strict CP system. This is a business decision, not a technical one.

User Sessions and Authentication Tokens → AP (usually)

A user's session token does not need to be perfectly consistent across all nodes within milliseconds. If a user logs in and gets routed to a node that hasn't received their new session yet, the worst case is a single extra login prompt.

Decision: AP with a short convergence window. DynamoDB with eventual consistency, or Cassandra. The availability gain (sessions always work) far outweighs the rare case of a stale token.

Exception: If you are implementing token revocation (forcibly logging out a compromised session), you need CP behavior for the revocation check — stale tokens that should be revoked remaining valid is a security risk.

Social Media Feeds and Engagement Counts → AP

A user's Twitter-like feed showing 1,249 likes instead of 1,250 is not a problem. Showing a post from 800ms ago instead of the absolute latest is not a problem.

Decision: AP. Cassandra or DynamoDB. These systems are optimized for exactly this workload. The consistency trade-off is invisible to users.

Distributed Locks and Leader Election → CP

If two workers both believe they hold a lock, they may both modify the same resource simultaneously — corrupting data or creating duplicate work.

Decision: CP, strictly. Use ZooKeeper, etcd, or Redis with Redlock (with caveats about Redlock's correctness). Never use an AP database for distributed locking.

Configuration and Feature Flags → CP

If half your nodes have old configuration and half have new configuration, your system may behave inconsistently in ways that are hard to debug.

Decision: CP. etcd or ZooKeeper. Configuration reads should never return stale data. The availability trade-off (if etcd is down, use last known config from local cache) is acceptable.

Frequently Asked Questions

What is the CAP theorem in simple terms?

The CAP theorem says that a distributed database can only guarantee two of three things at once: that every read gets the latest write (Consistency), that every request gets a response (Availability), and that the system keeps working even when nodes can't communicate (Partition Tolerance). Because network partitions happen in any real system, you always end up choosing between consistency and availability when a partition occurs.

Can a database be both consistent and available?

In a system that never experiences network partitions — like a single-node database — you can be both consistent and available (CA). But in any distributed system running across multiple machines or data centers, network partitions are inevitable. When a partition occurs, you must choose one: either stop serving requests to stay consistent (CP), or keep serving requests and accept potentially stale data (AP). You cannot guarantee both simultaneously during a partition.

What is the difference between CP and AP systems?

A CP system (Consistency + Partition Tolerance) refuses to serve inconsistent data. When a network partition occurs, it will return errors or block requests rather than risk returning different answers from different nodes. ZooKeeper, etcd, HBase, and PostgreSQL with synchronous replication are CP systems.

An AP system (Availability + Partition Tolerance) keeps responding during partitions, even if it means different nodes return different values temporarily. When the partition heals, the nodes sync up (eventual consistency). Cassandra, DynamoDB (by default), CouchDB, and Riak are AP systems.

Is Cassandra AP or CP?

Cassandra is AP by default. It prioritizes availability — it will always accept writes and serve reads, even during a network partition, at the cost of potentially returning stale data. However, Cassandra's tunable consistency allows you to shift toward CP behavior per query. Using ConsistencyLevel.QUORUM for reads and writes forces Cassandra to wait for a majority of replicas to agree, which sacrifices some availability for better consistency.

Is PostgreSQL consistent or available in CAP theorem?

It depends on your replication configuration. A single-node PostgreSQL instance is CA (trivially consistent, no partition possible). PostgreSQL with asynchronous replication leans AP — the primary stays available and the standby may lag. PostgreSQL with synchronous replication (synchronous_commit = on) is CP — writes only succeed when the standby confirms receipt, and if the standby is unreachable, writes block. For most production setups requiring strong consistency, synchronous PostgreSQL behaves as a CP system.

What is the PACELC theorem and is it better than CAP?

PACELC extends CAP to cover normal operation (not just partitions). It states: during a Partition, choose between Availability and Consistency; Else (normal operation), choose between Latency and Consistency. PACELC is more useful in practice because most of the time your system is not experiencing a partition — and the latency vs. consistency trade-off affects every single request. A system that guarantees strong consistency must wait for all replicas to confirm writes, which adds latency. A system that acknowledges writes immediately (low latency) cannot guarantee all replicas are consistent. PACELC makes this always-present trade-off explicit, whereas CAP only addresses the partition failure scenario.

How do I choose between consistency and availability for my system?

Start by asking what the worst-case consequence of inconsistency is. For financial data — balances, payments, inventory — the consequence of serving wrong data is financial loss or fraud, so choose CP. For user-facing features where "slightly stale" is invisible — feeds, profiles, counts, sessions — choose AP to maximize availability and performance. For distributed locks and coordination, always choose CP. A common pattern in large systems is to use CP databases for authoritative state (money, inventory) and AP databases for derived or user-facing state (feeds, caches, analytics). The two coexist; you are not making one choice for your entire architecture.

Key Takeaways

• CAP theorem: Consistency, Availability, Partition Tolerance — pick two. In practice, partition tolerance is mandatory, so the real choice is CP or AP.
• CP systems return errors during partitions rather than serve inconsistent data (ZooKeeper, etcd, HBase, PostgreSQL with sync replication)
• AP systems stay available during partitions but may return stale data (Cassandra, DynamoDB, CouchDB)
• PACELC extends CAP: even without partitions, every system trades latency for consistency
• Consistency exists on a spectrum: linearizable → sequential → causal → eventual
• Different parts of your system have different requirements — use CP for money and locks, AP for feeds and sessions
• Most web applications work well with AP + eventual consistency for the majority of their data

CAP is a fundamental constraint, not a design flaw. Understanding it means you stop asking "which database is best?" and start asking "what does my application need to tolerate?"

Related reading: Vertical vs Horizontal Scaling · Database Sharding Explained