Rate Limiting & Backpressure
The problem
A server has finite capacity, but clients can send requests faster than it can handle them, whether by accident (a buggy retry loop) or on purpose (abuse) — without a deliberate policy for rejecting or slowing excess requests, the server degrades for everyone instead of just the offending client.
Why now
CPU scheduling already established that a scarce resource needs an explicit allocation policy when demand exceeds supply; rate limiting is that same problem at the request-admission level instead of the CPU-instruction level — middleware and request pipelines already gave the mechanism (a pipeline stage) this policy plugs into.
Mental model
Rate limiting is admission control: before doing any real work, ask 'has this client already used its allotted budget in this time window,' and reject if so — protecting the server the same way a scheduler protects the CPU from being monopolized by one process. Backpressure is the same idea pushed further upstream: signaling 'slow down' to a producer instead of silently dropping or endlessly queueing what it sends.
Requires
- Middleware & Request PipelinesBackend Engineering
Rate limiting is implemented as a pipeline stage that runs before the expensive work, rejecting requests early exactly the way middleware-and-request-pipelines already described stopping a chain short.
- CPU SchedulingOperating Systems
Rate limiting is fundamentally the same problem cpu-scheduling already solved — allocating a scarce, contended resource fairly under demand that exceeds supply — just applied to incoming requests instead of CPU time.
Used in
- Public API rate limits (requests per minute per API key)
- Protecting login endpoints from brute-force attacks via aggressive rate limiting
- Backpressure in streaming systems, signaling a fast producer to slow down for a slower consumer
- 429 Too Many Requests as the standard HTTP response for a rate-limited client
Projects
- Implement a token-bucket rate limiter as middleware, allowing bursts up to a limit but refilling gradually over time
- Simulate a client exceeding the rate limit and confirm it receives a 429 with a clear Retry-After header
Examples
- A login endpoint limited to 5 attempts per minute per IP makes brute-force password guessing impractically slow
- A 429 Too Many Requests response with a Retry-After: 30 header tells a well-behaved client exactly how long to wait before retrying
Resources
Mastery checklist
- I can implement a basic rate limiter (fixed window or token bucket) as middleware
- I can explain why 429 with a Retry-After header is better than silently dropping requests
- I can explain backpressure and how it differs from simple rate limiting
- I can identify an endpoint in a real API that most needs rate limiting, and why