ZooKeeper

Apache ZooKeeper is a high-performance coordination service built for distributed systems. It presents a simple abstraction — a tree, much like a filesystem — and the coordination problems are solved on top of it: leader election, configuration, naming, group membership and distributed locks.

Coordination is the part of a distributed system that is hardest to get right and least rewarding to reimplement, which is why systems delegate it to a service designed for the purpose rather than writing it again per application. ZooKeeper remains an active Apache top-level project.

Znodes

A node in the ZooKeeper tree is a znode. Two kinds matter:

• Persistent — survives the session that created it.
• Ephemeral — deleted automatically when the session that created it ends.

Ephemeral znodes are what make failure detection work: a process that dies loses its session, its znode disappears, and everyone watching that part of the tree finds out.

Leader election

Electing a leader needs a registry mechanism and an election algorithm. ZooKeeper provides the first and the second falls out of it:

1. Every node that volunteers to become leader creates a znode under an /election parent. ZooKeeper names each one according to the order in which it was added.
2. The znode with the smallest sequence number is the leader. Each node queries the /election parent to learn about the nodes created before it.
3. A node whose own znode carries the smallest number knows it is the leader. A node that finds a smaller one knows it is not, and waits for instructions from the leader.

Because the volunteer znodes are ephemeral, the leader’s znode disappears when the leader dies, and the next-smallest node takes over.

A naive implementation has every node watch the /election parent, so the loss of one node wakes all of them at once — the herd effect. The usual fix is for each node to watch only the znode immediately ahead of it in the sequence, so a failure wakes exactly one process.

Watchers and triggers

A watcher delivers a one-shot notification when something changes. A watcher is registered by passing it to the call that reads the state being watched:

getData(znodePath, watcher)

Notifies when the znode’s data is modified.

getChildren(znodePath, watcher)

Notifies when the list of that znode’s children changes.

exists(znodePath, watcher)

Notifies when the znode is created or deleted.

A watcher fires once and is then gone; re-registering it on each notification is the caller’s responsibility, and the state may have changed again between the notification and the re-read.

Further reading

• Apache ZooKeeper — project site, documentation and current releases