Scalability Patterns
The problem
A system designed for its current traffic will eventually face more than it can handle, and the architectural decisions that make scaling easy or painfully hard are made long before that traffic arrives — this topic is about designing for growth deliberately, rather than discovering the bottleneck under pressure.
Why now
Load balancing and auto-scaling already established the infrastructure mechanisms for running many instances of a service; scalability patterns is the architectural discipline of designing services so those mechanisms actually work — which mostly comes down to one property: whether a service can be freely duplicated without breaking anything.
Mental model
The single biggest scalability decision is statelessness: a stateless service keeps no data specific to one instance, so a load balancer can send any request to any instance interchangeably, and adding capacity is as simple as adding more instances. The moment a service holds instance-specific state (an in-memory session, a local file), scaling it horizontally breaks unless that state is deliberately externalized — which is exactly the problem stateless design avoids from the start.
Requires
- Load BalancingCloud & DevOps
Scalability patterns are largely about making a service safe for a load balancer to distribute traffic across many interchangeable instances of; you need to understand what a load balancer requires (interchangeable backends) before designing a service to satisfy that requirement.
- Auto-ScalingCloud & DevOps
Designing a service to scale well is directly in service of auto-scaling actually working — a stateful service that breaks when instances are added or removed defeats auto-scaling regardless of how well-configured the scaling policy is.
Used in
- Twelve-factor app principles, several of which are directly about statelessness and horizontal scalability
- Session storage design — moving session data out of process memory and into a shared store (Redis) so any instance can serve any user
- Horizontal vs. vertical scaling as a fundamental architectural choice
- Identifying accidental state (in-memory caches, local file writes) that silently breaks horizontal scaling
Projects
- Take a service storing session data in local memory, and refactor it to store sessions in a shared external store so any instance can handle any request
- Explain, for a specific real service, what would break if it were suddenly run as 5 instances behind a load balancer instead of 1, and fix each issue
Examples
- A service storing user sessions in local memory breaks under load balancing — a user's second request might hit a different instance that's never heard of their session
- A truly stateless service can be scaled from 2 instances to 20 with zero code changes, because no instance holds anything the others don't already have equal access to
Resources
- The Twelve-Factor Apparticle
Mastery checklist
- I can explain why statelessness is central to horizontal scalability
- I can identify hidden instance-specific state in a service that would break horizontal scaling
- I can refactor a stateful service to externalize its state to a shared store
- I can explain the difference between horizontal and vertical scaling and when each is appropriate