Load Balancing
The problem
A single server can only handle so much traffic before it becomes a bottleneck or a single point of failure — load balancing spreads incoming requests across multiple servers, so capacity can grow by adding servers and no single server's failure takes the whole service down.
Why now
Distributed system basics already established that multiple machines can share work despite the possibility of any one failing; load balancing is the specific, practical mechanism that decides which machine handles which request, sitting in front of exactly the kind of socket-level connections sockets and ports already covered.
Mental model
A load balancer is a traffic router standing in front of a pool of identical servers, using a strategy (round-robin, least-connections, or based on server health) to decide which one gets each incoming request — and because it also detects unhealthy servers and stops sending them traffic, it doubles as the mechanism that turns 'one server died' from an outage into a non-event.
Requires
- Sockets & PortsNetworking
A load balancer accepts connections on a socket and forwards them to one of several backend sockets; you need the connection-endpoint model already established before 'routing a connection to one of several servers' is a well-defined operation.
- Distributed System BasicsDistributed Systems
Load balancing exists specifically because a system spans multiple machines that can independently fail; you need that partial-failure reality already established before health-check-based routing away from a failed server makes sense as a design, not just an optimization.
Unlocks
Used in
- Every production web service running more than one server instance
- Nginx, HAProxy, and cloud load balancers (ALB, ELB) as common implementations
- Health checks, automatically removing an unhealthy server from rotation
- Zero-downtime deployments, routing traffic away from servers being updated
Projects
- Run two instances of a simple web server locally behind a load balancer (Nginx or similar), and confirm requests are distributed between them
- Configure a health check that removes a server from rotation when it stops responding, and demonstrate traffic automatically avoiding it
Examples
- A round-robin load balancer sends the 1st request to server A, the 2nd to server B, the 3rd back to server A, and so on
- A load balancer's health check failing for one server stops routing new traffic to it within seconds, without any human intervention
Resources
Mastery checklist
- I can explain at least two load balancing strategies (round-robin, least-connections)
- I can configure a basic load balancer distributing traffic across multiple server instances
- I can explain how health checks let a load balancer route around a failed server
- I can explain how load balancing enables zero-downtime deployments