Distributed Systems using DDD

A distributed system is a set of processes running on different machines, communicating over a network, sharing state or working together towards a common goal.

Distributing a system is how complex products keep evolving without every change becoming a whole-system change. The recurring failure is not technical: spreading business logic across applications tends to destroy the shared understanding of the domain, and the bill arrives as developer time lost to re-learning, integration problems, and risky production deployments.

Domain-Driven Design (DDD) exists to give an organisation that shared understanding, and to isolate parts of it so that several teams can keep supporting the domain without depending on one person’s availability. The design is intended to help people understand the product and talk about it; that it also produces service boundaries is a consequence, not the goal.

Knowledge crunching

The modelling work itself is called knowledge crunching: discovering the concepts the domain requires by working through it with the people who know it. It involves analysts and developers together. Between them they add knowledge to a shared pool, then filter out everything that does not bear on the design need in front of them. The output is a model, and a vocabulary both sides use without translating.

Placing the boundaries

The hardest problem in a distributed system is that a transaction cannot span components and APIs freely. Two DDD concepts decide where the boundaries fall.

Aggregates mark transactional boundaries explicitly. An aggregate is the unit within which a change must be atomic.

Service APIs should be designed to map onto an action on an aggregate, which in turn maps onto something the end user believes they are doing. A service API can then be read as a set of operations, each a command to an aggregate, and each service API call is transactional.

Bounded Contexts group service APIs more broadly. A bounded context or subdomain may contain several aggregates.

Scaling a distributed system covers what happens to the storage layer once those boundaries exist, and ZooKeeper covers the coordination service that several of these systems lean on.

Eventual consistency

Where a distributed transaction across the boundaries is not viable — and two-phase commit, the classic strong-consistency option, holds locks across participants for the length of the exchange and scales badly as the path lengthens — the alternative is to accept eventual consistency and coordinate with messages instead of locks. Protocols for reaching it include:

Saga — a sequence of local transactions, each triggering the next by publishing an event, with compensating transactions to undo the earlier steps when one fails.
GRIT — eBay’s protocol for transactions spanning several databases, which collects read and write sets before any global commit decision, avoiding pessimistic locking across the commit.

Three service APIs, each issuing a command to its own aggregate inside one bounded context; each aggregate holds its own transactional model, so consistency between them is reached after the fact rather than inside one transaction.

Three further pages build on that footing. CQRS separates the model that writes from the model that reads. Event Sourcing keeps the event sequence itself as the record of truth. Reactive Systems states the properties the resulting system should hold.