Apache ZooKeeper
A coordination service that distributed systems use to agree on shared state such as configuration and leadership.
Apache ZooKeeper is a service that distributed applications use to coordinate. It stores a small amount of shared data in a tree structure and guarantees that every client sees the same view of it. Distributed systems use it for tasks such as knowing which node is the leader, tracking which members are alive and holding shared configuration.
The problem ZooKeeper exists to solve
Distributed systems keep running into the same handful of questions.
Which node is currently in charge? Which nodes are alive right now? Who holds the lock on this resource? What is the agreed value of this setting?
Each of these is deceptively hard. Answers that appear to work fail in the cases that matter: a network partition, a node that is slow rather than dead, two nodes that both believe they are the leader.
ZooKeeper is a service that answers these questions properly, so that every system needing them does not have to solve them again.
The data model
ZooKeeper stores data in a tree. Each node in that tree is addressed by a path, much like a file system, and holds a small value.
Small is the operative word. This is coordination data: which node is the leader, what the current configuration is, which members have registered. It is not a place for application data.
Ephemeral nodes
A node can be created as ephemeral, meaning it is tied to the session of the client that created it. If that client disconnects or stops, the node disappears without anyone removing it.
This is a neat solution to membership tracking. Each instance of a service creates an ephemeral node when it starts. The set of ephemeral nodes present is the set of instances currently alive. No heartbeat protocol is needed, because the disappearance of the node is the signal.
Watches
Rather than checking for changes repeatedly, a client can set a watch on a node and be notified when it changes.
A watch fires once, and the client sets a new one if it wants to continue being told. That design keeps the service from having to track an unbounded set of long lived subscriptions.
The ensemble
ZooKeeper runs as a group of servers, called an ensemble. One acts as leader and the others follow it.
Writes go through the leader and are agreed across the ensemble before being confirmed. The service continues working while a majority of servers are available, which is why deployments use an odd number: it gives the best tolerance for the number of machines involved.
What the guarantees mean in practice
The guarantee that matters most is ordering. Updates are applied in the same order everywhere, and a client's own operations are seen in the order it issued them.
That is what makes leader election possible. Without a consistent order, two nodes can each believe they won, and a system with two leaders will corrupt whatever it is managing.
Who works with ZooKeeper
Very few people use ZooKeeper directly. It is a dependency of other systems, and most encounters with it come from operating a platform that requires it.
Where it is used directly, it is by engineers building distributed systems that need coordination primitives.
Points to be aware of
It is another system to run. An ensemble has to be deployed, monitored, backed up and upgraded, and when it is unhealthy the systems depending on it are unhealthy too.
Storage layout also deserves attention. The documentation covers where the transaction log and snapshots are placed, and putting the transaction log on its own device is one of the more consistently useful pieces of guidance it offers.
Several projects that historically depended on ZooKeeper have since introduced their own built in coordination, so whether it is required depends on the version of the system you are running.
Getting started
The project documentation includes an overview, a getting started guide that runs a single server, an administrator guide covering ensemble configuration and storage, and programmer guides describing the client interfaces and the recipes for locking and leader election.
Key features of Apache ZooKeeper
Capabilities described in the official documentation.
A tree of small data nodes
Data is held in nodes arranged like a file system path, each holding a small value that clients read and write.
Ephemeral nodes
A node can be tied to a client session so that it disappears automatically when that client goes away.
Watches
A client can ask to be notified when a node changes, instead of checking for changes repeatedly.
An ensemble with a quorum
The service runs across several servers and continues operating as long as a majority of them are available.
Advantages of Apache ZooKeeper
Factual advantages that follow from the features above.
Every client sees the same state
Ordering guarantees mean clients do not act on different versions of the shared state at the same time.
Failure detection comes for free
Because ephemeral nodes vanish when a session ends, membership tracking does not need a separate heartbeat mechanism.
Hard problems solved once
Leader election and distributed locking are difficult to implement correctly, and are provided here as building blocks.
Clients are told when values change
A watch notifies a client that a node has changed, so shared configuration does not have to be polled.
Common use cases for Apache ZooKeeper
Situations the official documentation describes this tool as being used for.
Electing a leader
Several instances of a service use ZooKeeper to agree which one is currently in charge of a piece of work.
Tracking which members are alive
Each node registers an ephemeral entry on joining, so the current membership is visible without polling.
Holding shared configuration
Settings that every node must agree on are stored centrally, with watches notifying clients when they change.
Coordinating access to a shared resource
A distributed lock is used so that only one instance acts on a resource at any moment.
Official website
Everything on this page is based on the official documentation for Apache ZooKeeper. You can read the source here.
Apache ZooKeeper official documentation