Load Balancer
A load balancer distributes incoming requests across a pool of servers and returns each response to the client that asked for it. It exists for two reasons: to spread work that one server cannot handle alone, and to remove that server as a single point of failure by detecting failures and routing around them.
Load balancers come as dedicated hardware appliances and as software running on general-purpose hosts or as a managed service. The strategies and the layer distinction below apply to all of them.
Balancing strategies
Section titled “Balancing strategies”| Strategy | How it works |
|---|---|
| Round robin | Each server takes one request in turn. Distribution is equal by request count; actual server load is not measured. |
| Weighted round robin | Each server is given a weight that determines its share of the traffic. Useful when the pool is not homogeneous, and for releasing a new version gradually by sending it a small share first. Actual server load is not measured. |
| Source IP hash | The client’s source address is hashed to pick a server, so the same client keeps reaching the same one. This is how session state survives a page refresh or a dropped connection when the application holds state in memory. See sticky sessions. Actual server load is not measured. |
| Least connections | Servers with more open connections are assumed to be busier and receive fewer new requests. |
| Weighted response time | The balancer measures how long each server takes to answer a health check. A server that is busy with real traffic answers more slowly and is given a lower weight. |
| Agent-based | An agent on each server reports CPU utilisation, network throughput, disk operations and memory use, and the balancer weights on those readings. |
The first three distribute by rule and never look at the servers; the last three react to what the servers are actually doing. Rule-based strategies are cheaper and predictable; load-aware strategies cope better with requests whose cost varies widely.
Source IP hashing is the weakest of the session-persistence options: clients behind a shared NAT all hash to one server, and a client that changes network changes server. Where the application genuinely needs affinity, a Layer 7 balancer keyed on a cookie is more precise — and keeping session state out of the application servers entirely is better than either.
Layer 4 and Layer 7
Section titled “Layer 4 and Layer 7”The layer numbers are OSI’s — see the TCP/IP Network Model.
Layer 4 (transport). The balancer forwards TCP segments between the client and the chosen backend without interpreting what they carry. It reads addresses and ports, picks a backend when the connection opens, and then relays bytes. Overhead is low and the balancer can carry any TCP protocol, including encrypted traffic it holds no key for. It cannot route on anything inside the payload.
Layer 7 (application). The balancer terminates the connection, parses the HTTP request and chooses a backend on what it reads. That allows routing on:
- request path or host
- HTTP method
- content type requested
- cookies, including a session-affinity cookie the balancer sets itself
- any other request header
It also puts the balancer in a position to terminate TLS, compress responses, rewrite headers and serve cached content. The cost is the parsing itself, and the fact that the balancer must hold the TLS certificate and private key.
Load balancer or reverse proxy?
Section titled “Load balancer or reverse proxy?”A reverse proxy sits in front of one or more servers, takes requests on their behalf and returns their responses as though it had produced them. Every load balancer is therefore a kind of reverse proxy, but the reverse does not hold: a reverse proxy is worth deploying in front of a single server, whereas a load balancer only earns its place once there is a pool.
What a reverse proxy adds in front of one server:
- Isolation. Backend addresses are never exposed, so the backends cannot be reached directly and the infrastructure behind the proxy can be rebuilt without changing the address clients use.
- Edge enforcement. Rate limiting, IP denylisting, request size caps and WAF rules are applied once, at the edge, rather than in each backend.
- Offload. TLS termination, response compression and response caching are handled at the proxy, freeing backend capacity for application work. Caching in particular removes requests from the backends entirely.
In practice the two roles are usually filled by the same product — NGINX, HAProxy, Envoy, or a managed service — configured to do both. F5’s glossary entry on reverse proxies covers the distinction in more detail.