CockroachDB

The compromise it tries to avoid

For a long period the choice was framed as a trade.

Relational databases offered transactions and consistency, and scaling them meant a larger server or splitting the data by hand into pieces that no longer talked to each other properly.

Distributed databases offered scale across many machines, and typically restricted transactions to a single partition. Applications had to be written around that restriction.

CockroachDB is one of a set of databases built to avoid making that choice: distribute the data across nodes, and keep transactions and consistency working across all of them.

Whether this suits a given system depends on the specifics, but the ambition is what defines the category and the design.

Ranges

Data is divided into contiguous chunks of the key space called ranges.

Each range is replicated to several nodes. As data grows a range splits into two. As the cluster changes, ranges move between nodes so the distribution stays balanced.

This has consequences worth understanding.

Adding a node causes ranges to move onto it. Capacity grows by joining machines rather than by replacing hardware.

Losing a node removes some replicas, and other copies remain, so the data is intact and the cluster keeps serving. Replacement replicas are created elsewhere.

Nothing here requires an administrator to decide which data lives where by default. The cluster manages it, which is the part that manual sharding never handled well.

Transactions that cross machines

The difficult part is keeping transactions correct when the rows involved live on different nodes.

A transaction moving a value from one row to another must apply completely or not at all, whichever nodes hold those rows and whatever fails midway.

Achieving this requires nodes to coordinate and agree. That coordination is what makes the guarantee possible and it is also its cost, because agreement takes network round trips.

Within one data centre those round trips are short. Across regions, physical distance sets a floor on how fast agreement can happen, and no software removes it.

This is why data placement matters so much here. A transaction whose rows all live in one region is considerably faster than one spanning continents, and structuring data so that related rows sit together is the main performance consideration.

Speaking PostgreSQL

CockroachDB accepts the PostgreSQL wire protocol.

Practically this means existing PostgreSQL drivers connect to it, and many tools that work with PostgreSQL work here too. SQL that a developer already knows largely applies.

It is important to be precise about what this is. It is compatibility at the protocol and language level, not the same database. The internals are different, some features are unsupported, and behaviour differs in places. The official documentation maintains the list, and consulting it before assuming an application will move unchanged is the sensible order of operations.

Deciding where rows are stored

Because the cluster can span regions, where a particular row is stored becomes something worth controlling.

Placement rules can require rows to be held in specified regions.

Two reasons drive this. Requirements about where records may be held are common in regulated settings, and placing data near the users who read it reduces the distance requests travel.

Being able to express this as a rule in the database, rather than by operating separate databases per region and synchronising them, is one of the more practical capabilities of this design.

Who uses it

CockroachDB is used by engineering teams building applications that need to span regions, that must survive infrastructure failure without manual intervention, or that are outgrowing a single relational server while still depending on transactions.

Points to consider

CockroachDB is offered under source available terms and as a managed cloud service. The official website is the reference for licensing and for what each offering includes.

Distribution has a latency cost that is physical. Coordination across regions cannot be faster than the network between them, and expectations set by a single node database will not carry over unchanged.

It is more machinery than most workloads require. A single well configured PostgreSQL server handles a very large amount of work, and adopting a distributed database without a requirement for distribution adds operational complexity for no gain.

PostgreSQL compatibility should be verified against the specific application rather than assumed. The differences are documented precisely because they exist.

Getting started

The official documentation covers the architecture, SQL support, differences from PostgreSQL and data placement. Running a small local cluster and stopping a node while queries continue demonstrates the replication behaviour more convincingly than any description of it.