Caching, Explained: Make slow things feel instant.
Caching is the most powerful performance trick in software — and the easiest one to get subtly wrong. This guide takes you from zero to production-ready: every type, every pattern, every failure mode, with live simulators and C# examples.
What is caching?
A cache is a small, fast storage area that keeps copies of data so future requests for that data can be served faster than going to the original source.
A chef keeps salt, oil and chopped onions on the counter (the cache) instead of walking to the storeroom (the database) every time. The counter is small, so the chef only keeps what's used often. If the storeroom stock changes, the counter copy can be out of date — that's the central trade-off of caching.
⚡Speed
Memory reads are thousands of times faster than a disk-backed database query or a network call.
🛡️Protection
Absorbs traffic spikes so your database, or a rate-limited third-party API, isn't overwhelmed.
💰Cost
Fewer database servers, fewer paid API calls, less CPU spent recomputing the same result.
Hit and miss — the two outcomes
Every cache lookup ends in one of two ways. A cache hit means the data was found and returned fast. A cache miss means it wasn't there, so the app goes to the slow source, then usually stores the result for next time.
Your first cache
Request a product. The first request is a miss (slow). Request it again — now it's a hit. Try "Clear cache" to start cold.
Why it works: the latency ladder
Computers have a hierarchy of storage. Each step down is bigger and cheaper — but slower, often by 10× to 1000×. Caching copies hot data one or more steps up the ladder. Numbers below are rough orders of magnitude (log scale), not benchmarks.
Caching works because real traffic is lopsided. A small share of items (today's deals, a viral post, the logged-in user's profile) gets most of the requests — often called the 80/20 or power-law pattern. Keep that small hot set in fast memory and most requests become hits.
What's worth caching?
✅Great candidates
Read often, change rarely, expensive to produce, and OK if a few seconds stale: product catalogs, configuration, user profiles, rendered pages, aggregated dashboards, third-party API responses, permission lookups.
⛔Poor candidates
Changes on every read, read only once, must be exactly correct right now (account balance at transfer time, seat inventory at checkout), very cheap to fetch anyway, or highly personal and unbounded.
The cache landscape
A single web request can pass through many caches before it reaches your database. This course focuses on the backend and distributed layers, but you should know where the others fit.
The journey of one request
Tap a layer to learn about it, or press "Send request" to watch a request fall through the layers until one of them has the answer.
The closer to the user a cache sits, the faster it is — but the harder it is to invalidate. You can delete a Redis key instantly; you can't reach into a user's browser to remove a copy you told it to keep for a day.
Core vocabulary & the math
Learn these words once and every caching article, docs page and design interview becomes easier.
🎯Hit ratio
Hits ÷ (hits + misses). The headline health metric. 95% means 1 in 20 requests reaches the source.
⏳TTL (time to live)
How long an entry may live before it expires. Your main knob for trading freshness against load.
🔑Cache key
The unique name for an entry, e.g. product:42. Bad keys cause wrong data or wasted memory.
🧹Eviction
Removing entries because the cache is full — decided by an eviction policy such as LRU or LFU.
♻️Invalidation
Removing or updating an entry because the source data changed. Famously one of the two hard problems in computer science.
🥶Cold / warm cache
A cold cache is empty (after a deploy or restart), so everything misses. A warm cache already holds the hot set.
🍞Stale data
A cached copy that no longer matches the source. Some staleness is acceptable; the question is how much.
🔥Hot key
A single key that gets a huge share of traffic — a celebrity's profile, the live score of a final.
📦Serialization
Turning objects into bytes to store them in a remote cache (JSON, MessagePack, protobuf). It costs CPU and size.
🗂️Working set
The set of data actively used in a time window. If your cache can't hold the working set, the hit ratio collapses.
The one formula you need
Average latency = hit ratio × cache latency + (1 − hit ratio) × (cache latency + source latency). A miss costs the cache check plus the trip to the source. Play with it — notice how the last few percent of hit ratio matter enormously.
What does my hit ratio buy me?
Types of backend caches
"Add a cache" can mean very different things. The biggest decision is where the cache lives relative to your application process.
Same 200 requests, four architectures
Each lane serves 200 requests for product pages across a small set of popular products. Bars show total time spent waiting. Change the share of hot traffic and race again.
Read & write patterns
A caching pattern answers two questions: who loads the cache on a read? and what happens to the cache on a write? Pick a pattern and step through it.
Cache-aside
Patterns at a glance
| Pattern | Read path | Write path | Consistency | Best for |
|---|---|---|---|---|
| Cache-aside | App checks cache, loads DB on miss, fills cache | App writes DB, then deletes the key | Eventual (bounded by TTL) | The default for most read-heavy apps |
| Read-through | Cache library loads DB on miss | Usually paired with write-through or invalidation | Eventual | Cleaner code; HybridCache works this way |
| Write-through | Always a hit for written data | Write cache and DB together, synchronously | Strong-ish (both updated) | Data read right after write: profiles, carts |
| Write-behind | Hit | Write cache now, flush to DB later in batches | Weak; risk of loss | Counters, views, likes, telemetry |
| Write-around | Miss after a write | Write DB only; cache filled on next read | Eventual | Data written often, read rarely (logs, uploads) |
| Refresh-ahead | Hit (refreshed before expiry) | Background job reloads hot keys | Bounded staleness | Predictable hot keys: home pages, rates |
Expiration & TTL
Expiration is your safety net. Even if invalidation fails, a TTL guarantees stale data eventually disappears. Always set one.
⏰Absolute expiration
Dies at a fixed time, no matter how often it's read. "Cache exchange rates for 60 seconds." Guarantees a maximum staleness.
🔁Sliding expiration
Each read resets the timer. "Keep a session alive for 20 minutes of inactivity." Popular items can live forever — so combine it with an absolute cap.
🎲Jitter
Add a small random amount to each TTL (say ±10–20%) so thousands of keys loaded together don't all expire in the same second.
🧩Per-field TTL
Redis 7.4 added TTLs on individual hash fields; Redis 8 added combined commands HSETEX, HGETEX and HGETDEL.
Absolute vs sliding expiration
Both keys have a 10-second TTL (time runs 1:1). Keep pressing "Read both keys" and watch what happens to each timer.
Why you add jitter
After a deploy, 300 keys are cached at the same moment with a 60-second TTL. The chart shows how many keys expire in each second — every expiry is a trip to the database.
Choosing a TTL
| Data | Typical TTL | Why |
|---|---|---|
| Live score, stock ticker, bid price | 1–5 s (or push updates) | Users notice staleness within seconds |
| Product price & stock badge | 30 s–5 min + invalidate on change | Display only; checkout re-reads the source of truth |
| Product description, images, categories | 1–24 h + invalidate on edit | Rarely changes, heavily read |
| User profile, permissions | 5–15 min + invalidate on change | Security changes must propagate quickly |
| Session | Sliding 20–30 min, absolute cap of hours | Expire on inactivity, but never live forever |
| Config / feature flags | 30–60 s, or push on change | Fast rollout and fast kill-switch |
| Expensive reports / aggregates | Minutes to hours, refresh-ahead | Recomputing costs real money |
| "Not found" results (negative cache) | Short: 30–120 s | Blocks repeated lookups of missing IDs, but must not hide new data long |
Eviction policies
Memory is finite. When the cache is full and a new item arrives, something must leave. The eviction policy decides who — and that choice can swing your hit ratio by tens of percent.
🕰️LRU
Least Recently Used. Evict whatever hasn't been touched for the longest time. The classic default; great when recent means likely.
📊LFU
Least Frequently Used. Evict the item with the fewest hits. Keeps long-term favourites, but adapts slowly when popularity shifts (needs decay).
🚶FIFO
First In, First Out. Evict the oldest insertion. Very cheap, ignores popularity.
🎲Random
Evict a random item. Surprisingly decent when all keys are roughly equally popular.
🕳️SIEVE modern
A FIFO queue plus one "visited" bit and a moving hand. Simpler than LRU, lock-free on hits, and beats LRU on most web traces (NSDI '24).
🧪S3-FIFO / W-TinyLFU
Filter out "one-hit wonders" before they pollute the main cache. S3-FIFO uses three FIFO queues; W-TinyLFU (Caffeine) uses a frequency sketch.
Watch eviction policies compete
Tap letters to request keys, or play a preset workload. Green = hit, red outline = the victim chosen for eviction.
Redis / Valkey eviction settings
Redis evicts according to maxmemory-policy once it reaches maxmemory. Redis's LRU and LFU are approximations: it samples a handful of keys (maxmemory-samples, default 5) rather than tracking a perfect order, which saves memory with nearly identical results.
| Policy | Evicts | Use when |
|---|---|---|
noeviction | Nothing — writes fail with an error when full | Redis is a primary data store, not a cache |
allkeys-lru | Least recently used of all keys | Good default for a pure cache |
allkeys-lfu | Least frequently used (with decay) | Stable long-term favourites |
allkeys-lrm Redis 8.6+ | Least recently modified — reads don't refresh it | Evict data nobody has updated, even if it's still read |
allkeys-random | Random key | Uniform access, cyclic reads |
volatile-lru / -lfu / -lrm / -random | Same, but only keys that have a TTL | Mixing cache keys and permanent keys in one instance (better: use two instances) |
volatile-ttl | Key with the shortest remaining TTL | Your code assigns short TTLs to good eviction candidates |
The volatile-* policies behave like noeviction if no keys have a TTL. A cache full of keys without expiry plus volatile-lru means writes start failing.
Invalidation & consistency
When the database changes, the cached copy becomes a lie. Invalidation is how you make the cache tell the truth again — and races between readers and writers are where the bugs hide.
🗑️Delete on write recommended
After updating the DB, delete the key. The next read reloads fresh data. Simple and safe from most races.
✏️Update on write
After updating the DB, write the new value to the cache. Saves a miss, but two concurrent writers can leave the older value in the cache.
🏷️Tag / group invalidation
Label entries with tags (product:42, category:shoes) and remove all entries with a tag in one call. HybridCache and output caching support this.
🔢Versioned keys
Put a version in the key (catalog:v17:…). Bump the version to invalidate everything at once; old keys age out via TTL.
📣Event-driven (pub/sub, CDC)
Publish "product 42 changed" on a message bus, or stream DB changes (change data capture), and let every service invalidate its own copies.
📡Server-assisted near cache
Redis 6+ CLIENT TRACKING remembers which keys a client read and pushes an invalidation message when they change (best over RESP3).
The classic stale-cache race
A slow reader and a fast writer touch the same product at nearly the same time. Step through to see how the cache ends up holding an old price — even though the code "invalidates correctly".
Always keep a TTL so the damage is bounded. For critical data use one of: versioned/compare-and-set writes (only store if your version is newer), "delayed double delete" (delete again a second or two after the write), or update the cache from the DB change stream instead of from application code.
Tag-based invalidation
Each entry carries tags. Invalidate a tag and every entry carrying it disappears in one call — no need to know every key.
Consistency levels in plain words
| Level | What users see | How to get it |
|---|---|---|
| Eventual | May see old data for up to TTL seconds | Cache-aside + TTL (most apps) |
| Read-your-writes | You always see your own changes immediately; others may lag | Invalidate/update on write, or bypass the cache for the writer for a few seconds |
| Bounded staleness | Never older than N seconds | Absolute TTL = N; refresh-ahead |
| Strong | Always the latest committed value | Usually: don't cache it, or read the source of truth at decision time |
Failure modes
Caches don't just make things fast — when they misbehave they can take your whole system down. Learn the four famous failures and their cures.
🐃Stampede (thundering herd)
A hot key expires; hundreds of requests miss at once and all hit the DB together.
🕳️Penetration
Requests for IDs that don't exist always miss and always hit the DB — often from bots or attacks.
🏔️Avalanche
Many keys expire together, or the cache cluster restarts, and the full load lands on the DB.
🔥Hot key & big key
One key gets so much traffic (or is so large) that a single cache node or the network saturates.
Cache stampede: 100 requests, one expired key
A popular key is about to expire. Rebuilding it takes a 300 ms database query. 100 requests arrive across 3 app servers. Pick a defence and run it.
Stopping penetration with a Bloom filter
A Bloom filter is a tiny bit array that answers "definitely not here" or "maybe here". Add real user IDs, then look up IDs — including fake ones an attacker might send. Fake IDs are rejected without touching the cache or DB.
The cures, summarised
| Problem | Cures |
|---|---|
| Stampede | Request coalescing / single-flight (HybridCache does this per process), distributed lock with a short lease, serve-stale-while-revalidate, probabilistic early refresh (XFetch), refresh-ahead for known hot keys |
| Penetration | Validate input, cache "not found" briefly (negative caching), Bloom filter in front of the DB, rate-limit abusive clients |
| Avalanche | TTL jitter, warm the cache before sending traffic, replicas and high availability, circuit breakers and fallbacks so the DB isn't hammered when the cache is down |
| Hot key | Add an L1 in-process layer in front of Redis, replicate the key across shards (score:final#1..#8) and pick one at random, read from replicas |
| Big key | Split large values (chunking), store a list of IDs instead of full objects, compress, use hashes so you can fetch individual fields |
| Cold start | Pre-warm the top N keys on deploy, roll out gradually, keep cache nodes separate from app deploys |
Distributed caching
One cache server eventually isn't enough — for memory, throughput or availability. Distributed caches split data across many nodes. The key question becomes: which node owns which key?
🧩Sharding
Split keys across nodes so each holds a part. Redis Cluster uses 16,384 hash slots spread over the primaries.
🪞Replication
Each shard has replicas. If a primary dies, a replica takes over; replicas can also serve reads for hot keys.
💾Persistence
Optional snapshots (RDB) or append-only logs (AOF) so a restarted node isn't completely cold.
Consistent hashing vs modulo hashing
With hash(key) % N, adding one node moves almost every key — a mass cache miss. With a hash ring, each key belongs to the next node clockwise, so only a fraction move. Add or remove nodes and compare.
Choosing a distributed cache engine (late 2026)
| Engine | License / owner | What stands out | Consider when |
|---|---|---|---|
| Redis 8.x | Tri-license: RSALv2, SSPLv1 or AGPLv3 (Redis Ltd.) | JSON, search, vector sets, time series and probabilistic types bundled in core; hash-field expiry commands; LRM eviction since 8.6 | You want the richest feature set, Redis Cloud/Enterprise support or active-active geo-replication |
| Valkey 9.x | BSD-3-Clause, Linux Foundation | Fork of Redis 7.2.4; same RESP protocol, works with existing clients; extras via valkey-json / -search / -bloom modules | You need a permissive open-source license; it's the default on many managed cloud offerings now |
| Garnet | MIT, Microsoft Research | Written in C#/.NET; RESP-compatible; strong throughput with many client connections; managed option in Azure Cosmos DB | .NET-heavy teams, very high connection counts; check command coverage first |
| Memcached | BSD | Simple, multi-threaded key-value strings; no persistence; basis of Netflix's EVCache | Pure, simple, huge-scale look-aside caching |
| Dragonfly | Source-available | Multi-threaded, Redis- and Memcached-compatible API | Vertical scaling on big machines |
| NCache | Open source + enterprise | .NET-native; ships a HybridCache provider with pub/sub sync | Windows/.NET shops wanting in-house clustering |
Redis moved away from BSD in March 2024; Valkey forked days later. Redis 8 (May 2025) added AGPLv3 as an option. By mid-2026 Redis was on 8.8 and Valkey on 9.1. Redis Software 7.2 reached end-of-life on 28 Feb 2026, forcing many teams to choose a path.
Operational concerns
- Network is the cost. A remote cache hit still costs a round trip (~0.2–1 ms in the same zone). Batch with
MGETor pipelining instead of looping over keys. - Serialization. Large JSON payloads cost CPU and bandwidth. Store only what you need, consider MessagePack or protobuf, and compress big values.
- Timeouts and fallbacks. The cache is an optimisation — if it's down or slow, time out quickly and go to the DB (behind a circuit breaker) rather than hanging.
- Security. Enable TLS and authentication/ACLs, never expose the port to the internet, and avoid caching secrets or full card data.
- Multi-region. Each region usually has its own cache; replicate invalidations, not every write. Netflix's EVCache replicates across regions so users get local reads.
Caching in .NET
ASP.NET Core gives you four building blocks. For new code on .NET 9/10, HybridCache is the recommended default: in-memory L1, optional distributed L2, stampede protection and tags behind one GetOrCreateAsync call.
| API | Where data lives | Stampede protection | Tags | Use for |
|---|---|---|---|---|
IMemoryCache | This process only | No (DIY) | No | Single server, simple lookups |
IDistributedCache | Redis/Valkey/SQL/… | No (DIY) | No | Shared byte[] storage, sessions |
HybridCache | L1 memory + optional L2 | Yes (per process) | Yes | Default for app data |
| Output caching | Memory or Redis store | Yes (request locking) | Yes | Whole HTTP responses |
Stampede protection coalesces concurrent misses inside one process — with 10 servers you can still get up to 10 DB calls for one key. Tag invalidation removes entries for the current node and L2, but other servers' L1 copies may live until LocalCacheExpiration, so keep the L1 lifetime short. Libraries like FusionCache add a backplane and fail-safe if you need more.
Design a good cache key
Good keys are predictable, namespaced and include everything that changes the result. Toggle parts and see the key.
Real-world scenarios
Theory clicks when you see it in context. Each card describes a realistic problem and the caching design that fits it. Filter by industry.
When to use which
Answer four questions and get a recommended cache type, pattern, expiration and .NET API for your situation.
Which cache should I use?
The quick decision matrix
| Situation | Cache type | Pattern | Notes |
|---|---|---|---|
| One server, small reference data | In-process (IMemoryCache / HybridCache L1) | Cache-aside / read-through | Set size limits |
| Many servers, shared app data | HybridCache L1 + Redis/Valkey L2 | Read-through + delete on write | Short L1, longer L2 |
| Same JSON/HTML for many users | Output caching (Redis store when scaled out) | Read-through at HTTP layer | Vary by query/headers; evict by tag |
| Sessions, carts, tokens | Distributed (Redis/Valkey) | Write-through | Sliding expiry + absolute cap |
| Counters, likes, views | Redis (INCR) | Write-behind | Flush in batches; accept small loss |
| Expensive aggregates / ML features | Distributed or precomputed store | Refresh-ahead | Background jobs |
| Very hot single key | L1 in front of L2 + key replicas | Refresh-ahead | Avoid one-node hotspots |
| Must be exactly correct now | Usually none | Read the source of truth | Cache display, not decisions |
Monitoring & best practices
A cache you don't measure is a cache you don't understand. These are the signals and habits that separate healthy production caches from incidents.
🎯 Hit ratio
Per key prefix, not just overall. Redis: keyspace_hits / (hits + misses).
🧹 Evictions
evicted_keys rising = cache too small or wrong policy.
⏱️ Latency p99
Average lies. Watch the tail for slow commands and big keys.
🧠 Memory
Used vs maxmemory, fragmentation, and replication buffers.
Hit ratio dropping while evictions spike means the working set no longer fits. Misses spiking while evictions stay flat points to a stampede, a bulk invalidation, or a cold start. Many expired keys with a low hit ratio suggests your TTLs are too short.
Production checklist
Tick what your system already does. Your score is saved only in this browser.
Common anti-patterns
🚫 Caching without TTL
One missed invalidation and the data is wrong forever.
🚫 Cache as the only copy
Unless you deliberately run it as a durable store, assume anything in the cache can vanish.
🚫 Caching per-user data in shared output caches
A classic data leak: user A sees user B's page. Vary by user or don't cache.
🚫 N+1 cache calls
Looping 100 single GETs costs 100 round trips. Use MGET / pipelining or cache the whole list.
🚫 Huge values
Multi-MB values block the server and the network. Split or slim them.
🚫 Hiding a slow query
A cache masks a missing index until the first miss storm. Fix the query too.
Glossary & sources
Sources used for the 2026 facts
- Redis docs — Key eviction (policies, LRM since 8.6)
- Redis docs — HGETEX (Redis 8.0)
- Redis blog — What is Valkey? (licensing, Redis 7.2 EOL)
- Redis vs Valkey versions, mid-2026
- Chat2DB — Valkey vs Redis (Sep 2026)
- Microsoft Garnet on GitHub
- HybridCache in ASP.NET Core .NET 10
- Milan Jovanović — HybridCache in ASP.NET Core
- Cache stampede prevention (X-Fetch, locks)
- Server-assisted client-side caching
- USENIX — SIEVE is simpler than LRU
- Harvard Systems — Why FIFO is (almost) all you need
- InfoQ — Netflix global caching with EVCache
- Netflix EVCache wiki
- Cache stampede mitigation patterns