Auto-Scaling
The problem
Provisioning enough servers for peak traffic means paying for idle capacity most of the time, while provisioning for average traffic means falling over during a spike — auto-scaling adjusts the number of running servers automatically based on real-time demand, instead of guessing a fixed number upfront.
Why now
Load balancing already established a pool of interchangeable servers sharing traffic; auto-scaling is the missing piece that grows or shrinks that pool automatically, and CPU scheduling already established the general principle of allocating resources based on current demand rather than a fixed static assignment.
Mental model
Auto-scaling is a feedback loop, just like Kubernetes' reconciliation loop: continuously measure a signal (CPU usage, request rate), compare it to a target, and add or remove servers to close the gap — the load balancer already covered is what makes this possible at all, since new servers need somewhere to automatically start receiving traffic the moment they're ready.
Requires
- Load BalancingCloud & DevOps
Auto-scaling only works because a load balancer can add a newly started server to rotation automatically; you need that routing mechanism already in place before scaling the pool size is a meaningful, safe operation.
- CPU SchedulingOperating Systems
Auto-scaling is resource allocation under changing demand, the same principle already covered for CPU time within one machine, generalized to entire server instances across a whole fleet.
Unlocks
Used in
- Cloud provider auto-scaling groups (AWS ASG, GCP managed instance groups)
- Kubernetes' Horizontal Pod Autoscaler, applying the same idea to container replica counts
- Cost optimization — paying only for capacity actually needed at any given moment
- Handling traffic spikes (a sale, a viral post) without manual, reactive intervention
Projects
- Configure an auto-scaling policy (real or simulated) that adds instances when CPU usage exceeds a threshold and removes them when it drops, and test it under simulated load
- Explain the tradeoff between an aggressive auto-scaling policy (fast to react, risk of thrashing) and a conservative one (slow to react, risk of under-provisioning)
Examples
- An auto-scaling group configured to add an instance when average CPU exceeds 70% can absorb a traffic spike within minutes instead of requiring a human to notice and react
- Scaling down too aggressively right after scaling up (thrashing) can cause repeated cycles of adding and removing capacity instead of settling
Resources
Mastery checklist
- I can explain how auto-scaling uses a metric and a threshold to decide when to add or remove capacity
- I can configure a basic auto-scaling policy
- I can explain scaling 'thrashing' and one way to prevent it (cooldown periods)
- I can explain the cost/reliability tradeoff auto-scaling policy tuning represents