Caching Strategies

Caching is storing the result of expensive work so you can reuse it instead of redoing it — the single highest-leverage performance technique in most applications. A query that takes 200ms against the database takes microseconds from memory. But caching introduces the hardest problem in computer science: invalidation — keeping the cache from serving stale data. This page covers the layers of caching (in-process, distributed, HTTP, CDN), the read/write patterns, TTL and eviction, the invalidation strategies, and the pitfalls (stampedes, stale data) that bite in production.

The caching layers — closer to the user is faster

Layer	Where	Latency	Scope
In-process (`Map`, LRU)	Inside the Node process	Nanoseconds	Per-instance — not shared
Distributed (Redis, Memcached)	Separate server	Sub-millisecond (network)	Shared across all instances
HTTP cache	Browser / proxy	Zero server work on hit	Per-client / shared proxy
CDN	Edge, near the user	Edge latency	Global, for static/cacheable content

In-process caches are fastest but per-instance — use Redis when you run multiple instances

The closer a cache sits to the request, the faster it is — but the trade-offs differ. An **in-process** cache (an LRU `Map` in your Node app) is the fastest possible lookup, but each instance has its own copy: with three instances behind a load balancer you get three caches that can disagree, and a cache invalidation on one doesn't reach the others. A **distributed** cache like [Redis](/nodejs/redis) adds a sub-millisecond network hop but is *shared* across every instance, giving one consistent view. Many systems layer both: a tiny in-process cache in front of Redis for the hottest keys. Push static content to **HTTP/CDN** layers so requests never reach Node at all.

Cache-aside — the default read pattern

async function getProduct(id) {
  const key = `product:${id}`

  // 1. Try the cache first:
  const cached = await redis.get(key)
  if (cached) return JSON.parse(cached)          // cache HIT

  // 2. Miss → load from the source of truth:
  const product = await db.products.findById(id)

  // 3. Populate the cache with a TTL, then return:
  await redis.set(key, JSON.stringify(product), 'EX', 300)   // expire in 5 min
  return product
}

Cache-aside: check cache, miss → load from DB → populate cache. Simple and the most common.

**Cache-aside** (lazy loading) is the workhorse pattern: the application checks the cache, and on a miss loads from the database and writes the result back. Only requested data is ever cached (efficient), and the database remains the source of truth. Its weaknesses: the *first* request for each key always pays the full cost (a cold cache), and data can go stale between writes and TTL expiry. The alternatives — **write-through** (write to cache and DB together, keeping them in sync at write cost) and **write-behind** (write to cache, flush to DB asynchronously, fast but risks loss) — trade complexity for freshness. Cache-aside is the right default unless you have a specific reason otherwise.

TTL and eviction

// In-process LRU with a size bound AND a time bound:
import { LRUCache } from 'lru-cache'
const cache = new LRUCache({
  max: 5000,          // evict least-recently-used past 5000 entries (bounds memory)
  ttl: 1000 * 60,     // entries expire after 60s (bounds staleness)
})

// Redis: every key gets an expiry — never cache without one:
await redis.set('session:abc', data, 'EX', 1800)   // 30-minute TTL

Always set a TTL and a size bound — an unbounded cache is a [memory leak](/nodejs/memory-leaks) and stale data forever

Two limits are non-negotiable. A **size bound** (LRU eviction) stops the cache growing until it exhausts memory — an unbounded in-process cache is the classic Node [memory leak](/nodejs/memory-leaks). A **TTL** (time-to-live) bounds how stale data can get and acts as a safety net for invalidation you missed — even if you forget to evict on a write, the entry self-destructs after the TTL. Choose the TTL by how tolerant the data is of staleness: seconds for fast-changing data, hours for near-static reference data. Caching without expiry guarantees you'll eventually serve wrong data and run out of memory.

Invalidation — the hard part

Strategy	How	Trade-off
TTL expiry	Let entries expire after N seconds	Simple; serves stale data up to the TTL
Explicit eviction	Delete the key when the data changes	Fresh; you must find every write path
Write-through	Update cache on every write	Always consistent; adds write latency
Versioned keys	Bump a version in the key (`product:42:v7`)	Old versions age out naturally; no deletes

On every write path, evict the affected keys — a forgotten path is a stale-data bug

Explicit invalidation is correct but fragile: you must evict (or update) the cached value on **every** code path that modifies the underlying data. Miss one — an admin tool, a batch job, a different service writing the same table — and users see stale data with no obvious cause. Strategies to reduce the risk: keep a TTL as a backstop so stale data self-corrects eventually; centralize writes through a single repository layer that owns invalidation; or use versioned keys so you never have to delete, just point at a new version. "There are only two hard things in computer science: cache invalidation and naming things" is a joke that bites for real.

The cache stampede problem

// ❌ When a popular key expires, hundreds of concurrent requests all miss at once
//    and hammer the database simultaneously (the "thundering herd").

// ✅ Single-flight: collapse concurrent misses into ONE database load:
const inFlight = new Map()
async function getProduct(id) {
  const key = `product:${id}`
  const cached = await redis.get(key)
  if (cached) return JSON.parse(cached)

  if (inFlight.has(key)) return inFlight.get(key)   // reuse the in-progress load
  const promise = db.products.findById(id).then(async (p) => {
    await redis.set(key, JSON.stringify(p), 'EX', 300)
    inFlight.delete(key)
    return p
  })
  inFlight.set(key, promise)
  return promise
}

When a hot key expires, concurrent misses can stampede the database — collapse them into one load

A **cache stampede** (thundering herd) happens when a popular cached key expires and, in the same instant, many concurrent requests all miss and all query the database at once — a traffic spike that can overwhelm the DB precisely when the cache was supposed to protect it. The fix is **single-flight** / request coalescing: track in-progress loads so concurrent misses for the same key await one shared database call rather than each firing their own. Complementary tactics include slightly randomized ("jittered") TTLs so many keys don't expire simultaneously, and serving slightly-stale data while one request refreshes in the background (stale-while-revalidate).

What to cache — and what not to

Cache: expensive reads that are read far more than written — product catalogs, config, computed aggregates, rendered fragments.
Cache: idempotent external API responses — to cut latency and respect rate limits.
Don't cache: per-user sensitive data in a shared cache without correct key scoping — you risk leaking one user's data to another.
Don't cache: rapidly-changing data where staleness causes real harm (live inventory, balances) unless TTL is tiny.
Measure the hit rate — a cache with a low hit rate adds complexity and latency for little gain.
Beware caching errors/empty results — a cached 404 or null can mask data that later appears.

Shrink what you send over the wire: [Response Compression](/nodejs/compression).