Caching Strategies on the Server
The problem
Recomputing or refetching the same expensive result for every request wastes resources and adds latency — server-side caching stores a result once and reuses it for subsequent requests, at the cost of the classic hard problem of knowing when that stored result is no longer valid.
Why now
Hash tables already gave the fast key-to-value lookup structure every cache is built on; CDNs and caching already introduced the freshness/staleness tradeoff at the network edge. This topic is that same tradeoff, applied inside the server itself, closer to the expensive computation being avoided.
Mental model
A cache is a hash table trading memory for speed: check the cache first using a key derived from the request, return the cached value on a hit, compute and store on a miss. Every caching strategy is really a policy for two decisions — how long is a cached value considered fresh, and what happens when the underlying data changes before that time is up.
Requires
- Hash TablesComputer Science Fundamentals
A cache is a hash table mapping a request or computation's key to its previously computed result; the O(1) lookup that makes caching worthwhile is exactly the hash table's core guarantee.
- CDNs & CachingNetworking
The freshness/staleness tradeoff (TTLs, invalidation) is identical to the one already covered for CDN and HTTP caching, just applied one layer closer to the server's own computation instead of at the network edge.
Used in
- Redis and Memcached as dedicated in-memory caching layers in front of databases
- Caching expensive database queries or third-party API calls
- Cache invalidation strategies (TTL, write-through, cache-aside) as a recurring architectural decision
- Understanding cache stampede — many requests recomputing the same expired value simultaneously
Projects
- Add a cache-aside layer in front of a slow function (simulate with an artificial delay), measuring the latency improvement on cache hits
- Implement TTL-based expiration for a cache entry and demonstrate a request correctly recomputing the value once it expires
Examples
- A cache-aside pattern checks Redis first, falls back to the database on a miss, then writes the result back to Redis for next time
- A cache stampede happens when a popular cached value expires and hundreds of simultaneous requests all recompute it at once instead of one request refreshing it for everyone
Resources
- Caching strategies — AWSarticle
Mastery checklist
- I can implement a cache-aside pattern in front of an expensive operation
- I can explain the tradeoff a TTL makes between freshness and cache hit rate
- I can explain what a cache stampede is and one strategy to prevent it
- I can choose an appropriate caching strategy given a data's update frequency