Caching Strategies

The big picture#

Everything below expands on these points. Read the core sections top to bottom for the full mental model; the collapsible "Go deeper" boxes hold the advanced internals (latency math, leases, tuning) you can skip on a first pass.

Start here: the problem it solves#

Your database can serve, say, a few thousand reads per second; your traffic wants a hundred thousand. A cache absorbs most of those reads from memory in well under a millisecond. But now the same data lives in two places, and the moment they can disagree you have a correctness problem layered on top of a performance win.

So caching is really a set of policy choices about three actors that must stay both fast and consistent: the application (which wants answers), the cache (a fast but ephemeral copy), and the backing store (the slow but authoritative source of truth). On a read miss, who loads the data and who populates the cache? On a write, do you update the cache, the store, or both — and in what order? How long does a cached entry live? And what happens when a popular key expires and a thousand requests miss at once?

Keeping the copy consistent with the source is the whole game. The cache only helps if it mostly hits; it only stays correct if writes and expiry keep its copy close enough to the store. Every strategy below is one answer to 'how do these three stay fast and agree?'

Read patterns: cache-aside vs read-through#

Cache-aside (also called lazy loading) puts the application in charge. The app checks the cache; on a miss it reads the store itself and then writes the value back into the cache so the next read hits. It's the most common pattern because it's simple and the cache and store stay decoupled — if the cache is down, the app just talks to the store directly (degraded, not broken). The cost: the loading logic lives in every caller, and there's a brief window where concurrent misses for the same key can each load it.

1. App asks the cache for key K.
2. Hit → return the cached value. Done in ~1ms.
3. Miss → app reads K from the backing store.
4. App writes K back into the cache (usually with a TTL).
5. App returns the value. Next read for K is a hit.

Read-through moves that logic into the cache layer. The application only ever talks to the cache; on a miss the cache itself loads from the store, populates, and returns the value. The behavior is similar to cache-aside, but the loading is centralized in one place (a cache library or provider) instead of duplicated across application code — and a single read-through cache can coalesce concurrent misses internally.

Write patterns: through, back, around, invalidate#

Read patterns decide who fills the cache; write patterns decide how a change reaches the store and what happens to the cached copy. There are four common choices, and they differ mainly in where the latency lands and what a crash can lose.

• Write-through — write the cache and the store synchronously, together, before acknowledging. The cache is always consistent with the store and reads are warm, but every write pays the store's latency (two hops). Best when reads soon follow writes and you can't tolerate stale cache.
• Write-back (write-behind) — write only the cache, mark the entry dirty, acknowledge immediately, and flush dirty entries to the store later in batches. Writes are very fast and the store sees far fewer, coalesced writes — but a crash before the flush silently loses every un-flushed write. Durability is sacrificed for throughput.
• Write-around — write straight to the store and skip the cache entirely. The cache fills only on a later read miss. Good for write-heavy data that's rarely read soon (logs, audit records): it avoids polluting the cache with entries nobody will read, but the first read after a write is always a miss.
• Write-invalidate — write the store, then delete the cached entry (rather than updating it) so a stale copy can't linger and the next read reloads fresh. This is the standard write path paired with cache-aside reads.

The recurring tension is consistency vs. speed vs. durability. Write-through is safe but slow; write-back is fast but risks loss; write-around and write-invalidate avoid serving stale data at the cost of more misses. Note these compose: write-through + read-through gives an always-warm consistent cache; cache-aside + write-invalidate gives a lazy cache that's eventually consistent within the TTL.

The numbers: latency, hit rate, staleness window#

A cache's payoff is governed by one equation. With cache hit latency h, store (miss) latency m, and hit rate r, the average read latency is: r·h + (1−r)·(m + h). The (m + h) on a miss reflects that a cache-aside miss pays the store and then the cache write. Because m is often 10–100× h, the hit rate r dominates everything.

• Hit rate is non-linear in value. Going from 90% → 99% hit rate roughly 10× reduces the share of requests that touch the store — often the difference between the database coping and falling over. The last few percent of hit rate matter most, because misses are what generate store load.
• Cache vs store latency: a RAM cache hit is ~0.1–1ms; a database miss is ~5–50ms; a cross-region or disk miss can be 100ms+. The whole point is to keep most reads on the fast path.
• Store load is what you're really protecting. If the store handles Q queries/sec and you receive N reads/sec, you need a hit rate of at least 1 − Q/N or the store saturates. A cache isn't just for latency — it's a shield that bounds backing-store QPS.

Variants: TTL expiry, refresh-ahead, negative caching#

• TTL-based expiry — each entry carries a time-to-live; after it elapses the entry is treated as absent and the next read reloads. The simplest way to bound staleness without explicit invalidation. Shorter TTL = fresher but lower hit rate; longer TTL = higher hit rate but staler.
• Refresh-ahead — proactively reload an entry that is hot and near expiry, before it actually expires, so reads never hit a cold miss. It trades extra background loads (some wasted, for keys that wouldn't have been read again) for lower tail latency on hot keys. Effective only when access is predictable.
• Negative caching — cache the fact that a key does NOT exist (a 'not found' sentinel) for a short TTL. Without it, every request for a missing key bypasses the cache and hits the store every time — the basis of cache penetration. A short negative TTL absorbs those misses while limiting how long a later-created key stays invisible.

The core tension: consistency vs latency vs durability#

Every strategy is a point in a triangle of consistency (does the cache agree with the store?), latency (how fast are reads and writes?), and durability (can an acknowledged write be lost?). You cannot maximize all three at once.

• Consistency vs latency: write-through and write-invalidate keep the cache close to the store but add a synchronous store hop to every write. Cache-aside with a long TTL is fast but can serve data that's stale by up to the TTL.
• Latency vs durability: write-back gives the fastest writes by acknowledging before the store is updated — but that un-flushed window is exactly what a crash loses. Write-through is durable because the store is updated before the ack.
• Consistency across replicas: when many app servers each hold their own cache, a write must invalidate all of them. That needs an invalidation message bus (or short TTLs as a crude substitute), and there's always a propagation delay during which different servers can disagree.

Failure modes: stampede, penetration, avalanche, loss#

Caches fail in characteristic ways, almost always by letting too much traffic reach the store at once. The three load-spike failures have specific names and specific fixes.

• Cache stampede (thundering herd) — a single hot key expires (or is evicted) and every concurrent request for it misses at the same instant, all stampeding the store with the same query. Fix: request coalescing / single-flight (one loader, the rest wait and share the result), a short lock or lease per key, or early/probabilistic recomputation that refreshes a hot key slightly before it expires.
• Cache penetration — requests for keys that don't exist anywhere bypass the cache (nothing to hit) and hammer the store on every request. Common with malicious or buggy clients probing random IDs. Fix: negative caching (cache the 'not found' for a short TTL) and/or a Bloom filter in front of the cache to reject keys that definitely don't exist.
• Cache avalanche — a large number of entries expire at the same moment (e.g. everything loaded at startup with the same TTL), so a huge wave of misses hits the store simultaneously. Fix: jitter the TTLs (add a small random spread) so expirations are smeared over time, and warm the cache gradually rather than all at once.
• Write-back data loss — with write-behind, every write acknowledged but not yet flushed is held only in cache memory. A crash, eviction, or restart before the flush loses it permanently and silently. Mitigate with a persistent write-ahead log for the dirty buffer, replication of the cache tier, or by reserving write-back for data you can afford to lose.
• Stale reads — any pattern that fills the cache and then lets the store change underneath it (long TTL, failed invalidation, replication lag) serves data older than the source of truth. Bound it with shorter TTLs, reliable invalidation, or versioned keys.

Strategies side by side#

Where these patterns run in the wild#

• Redis & Memcached — the two default in-memory caches. The cache-aside + write-invalidate combo on Redis is the most common application caching pattern. Redis adds server-assisted client-side caching (the cache tells clients when their cached keys change, so each app server can keep a tiny local copy and have it invalidated automatically).
• Facebook memcache — the canonical large-scale deployment. Cache-aside (they call it 'demand-filled look-aside'), leases to kill stampedes and stale sets, and a region-wide invalidation pipeline driven off the database commit log. Documented in 'Scaling Memcache at Facebook' (NSDI 2013).
• Netflix EVCache — a distributed, replicated caching tier built on Memcached, spanning AWS availability zones. Writes fan out to multiple zones for availability; it's a global read-through/write-aside layer tuned for very high hit rates and zone failure tolerance.
• CDNs (CloudFront, Fastly, Cloudflare) — caching at the network edge. They live and die by TTLs (Cache-Control headers), stale-while-revalidate, and request coalescing at each edge so origin servers see one fetch per object, not one per viewer.
• Database query/buffer caches — the buffer pool in PostgreSQL/MySQL is an internal write-through-ish cache of disk pages; query result caches and materialized views are application-visible caches with their own invalidation rules.

Common misconceptions & gotchas#

In an interview#

Frame it as three actors — app, cache, backing store — and two sets of choices. Name the read patterns (cache-aside / lazy loading, read-through) and the write patterns (write-through, write-back, write-around, write-invalidate), then reason about which fits the workload: read-heavy with tolerable staleness → cache-aside + TTL; write-then-read, must-be-fresh → write-through; write-heavy, rarely-read → write-around; throughput-critical, loss-tolerant → write-back with a persistent log. Say explicitly why you invalidate rather than update the cache (idempotent, avoids write races).

Then volunteer the failure modes — this is what separates a strong answer. Stampede on hot-key expiry → request coalescing / single-flight + TTL jitter. Penetration on missing keys → negative caching or a Bloom filter. Avalanche on mass expiry → jittered TTLs. Write-back loss → persistent log or replication. Bonus points for naming Facebook's leases (one mechanism that kills both stampedes and stale sets) and for noting the cross-server consistency problem (invalidation bus vs. short TTLs).

Then open the simulator: WRITE with different strategies and watch the cache and store agree or diverge, READ to see hits vs. miss-then-load, TICK to expire a TTL and flush write-back, CRASH to lose dirty data, and BURST a hot key with stampede protection on and off.

References & further reading#