Caching
Keep a copy of the answer near whoever keeps asking for it.
What problem does caching solve?
Databases are good at storing things. They are slow at answering the same question ten thousand times a second. A cache keeps that answer somewhere faster and closer, so most requests never reach the database at all.
In a read-heavy system it is often the biggest single win. It is also a common source of stale-data bugs.
What caching costs you
Every concept in here states a cost. A technique with no price is a technique you have not understood yet — and naming the cost out loud is most of what separates a senior answer from a junior one.
When to reach for caching, and when not to
Use it when
- Reads are far more common than writes. This is the main reason.
- The same items get asked for again and again (a popular product, a hot post).
- The answer is expensive to work out, and slightly old data is acceptable.
Do not, when
- The data must be correct at this instant: account balances at payment time, remaining seats, stock counts.
- Every request asks for something different. A cache that never gets a hit is pure cost.
- Writes happen almost as often as reads. You will spend all your time clearing the cache.
How caching actually works
Where to put it. In the browser: free and closest, but you cannot take it back once you have sent it. In a CDN: great for anything shared between users. Inside the application process: fastest of all, but every server has its own copy, so they drift apart. In a shared cache like Redis or Memcached: one copy every server agrees on, at the cost of a network hop. Inside the database itself (its buffer cache): it helps, but you do not control it.
Cache-aside is the default, and the one to describe. The application asks the cache. If it is not there, the app reads the database, writes the answer into the cache, and returns it. It is simple. And if the cache goes down you get slower, not broken.
Write-through writes to the cache and the database together. The cache is never stale, but every write pays both costs, and you also cache items that may never be read. Write-behind writes the cache now and the database shortly after. That is fast, and you can lose data in the gap. Say which one you picked, and why.
Invalidation has two common approaches, and both are compromises. A TTL means the data is wrong for at most that long, and you never have to think about it. Deleting the key on every write means it is right almost always, but every piece of code that writes has to remember. One path that forgets creates a stale-data bug that is hard to reproduce. Many systems use a TTL as the safety net and explicit deletion on top of it.
The empty-cache problem is worth knowing by name. On a cold start, or after a mass eviction, every request goes to the database at the same moment. The database falls over. And when it comes back, the cache is still empty. There are three defences. Request coalescing: when a hundred requests miss on the same key, let one go to the database and make the other ninety-nine wait for that answer. Jittered TTLs, so a million keys written together do not all expire in the same second. And warming the cache before you send traffic to it.
A related problem: a request for a key that does not exist anywhere hits the database every time, because there is nothing to cache. If that is common, say because someone is probing random IDs, cache the "not found" answer too, briefly. Or put a bloom filter in front.
- Saying "add a cache" without saying what clears it. That is the obvious follow-up question.
- Assuming a cache makes writes faster. It does not. It usually makes them slightly slower.
- Ignoring what happens when it is empty. That is where the outage lives.
The caching follow-up an interviewer will ask
"Your cache just died. What happens?"
Every read that was being served from memory now goes to the database at once. So the answer starts with a question: does the database survive that? Usually not, at the ratios that made me add a cache in the first place. So I would not depend on the cache being up. I would coalesce requests, so one miss on a key means one database read and not ten thousand. I would put a small in-process cache on each application server as a second layer, so even with the shared cache gone we absorb the hottest keys. I would rate-limit or drop traffic at the edge instead of letting the database die, because a slow site beats a dead one. And I would bring the cache back warm, not empty. The cost of all that: more moving parts, and the in-process layer means servers can briefly disagree with each other.
Your turn
Name one thing you should never cache in a booking system, and say why.
The cache that took down the database at 9am every Monday
Every Monday at 09:00 the site slows to a crawl for about four minutes, then recovers on its own. Database CPU hits 100%, application error rate spikes, and by the time anyone opens a dashboard it is already fixing itself. Redis is healthy throughout. This has happened for six weeks.
- 1Find what is special about 09:00 Monday
It is not traffic. Monday peak is only slightly above other days, and nothing like a fourfold spike. The clue is that Redis is healthy but the database is saturated, which means requests are missing the cache. So the question is not "why is there more traffic" but "why did the hit rate collapse".
- 2Graph the hit rate, not the traffic
Hit rate is 99.2% all week and drops to 61% for four minutes every Monday at 09:00. That is the incident: at 99.2% the database sees 0.8% of reads, and at 61% it sees 39%. That is close to fifty times the database traffic, with no change in user traffic at all.
- 3Find why the keys expired together
A weekly job repopulates product data on Sunday night, writing about two million keys with a fixed 12-hour TTL. They were all written within a few minutes of each other, so they all expire within a few minutes of each other, at roughly 09:00 on Monday. A synchronised mass expiry, created by the cache-filling job itself.
- 4Fix it in the cheapest place
Add jitter: the TTL becomes 12 hours plus a random 0 to 90 minutes. The same two million keys now expire spread over an hour and a half instead of over four minutes, and the database barely notices. It is one line of code, and it removes this incident.
- 5Then make the class of failure survivable
Jitter fixes this occurrence; it does not fix the next mass eviction. So also add request coalescing, so a thousand simultaneous misses on the same key become one database read. And add a short in-process cache in front, so even a completely empty Redis does not send full read load to the database. Now the database is protected from an empty cache generally, rather than from this specific Monday.
- 6Alert on the leading indicator
The alert was on database CPU, which fires when the damage is already happening. Add an alert on cache hit rate dropping below a threshold, which fires before the database is in trouble and points directly at the cause instead of at the symptom.
Learn it from the source
This page gets you interview-ready. These take you to where the idea is actually specified or measured — read them when you want the real depth.
- Redis — Key eviction ↗LRU, LFU and the rest as real policies you configure, not just interview words.
- RFC 9111 — HTTP Caching ↗The specification for TTLs, revalidation and stale-while-revalidate.