Caching

Caching stores frequently accessed data in fast storage to reduce latency and database load.

Performance Impact

Problem	Without Cache	With Cache
Database reads	10-100ms	1-10ms
Repeated computations	CPU cost per request	Computed once
API rate limits	Limits reached quickly	Reduced external calls
User experience	Slow page loads	Fast responses

Cache is most effective when data is read more frequently than written.

Cache Placement

Loading diagram...

Layer	Cached Content	Tools	Latency
Client	Static assets, API responses	Browser cache, localStorage	0ms
CDN	Images, CSS, JS, videos	CloudFront, Cloudflare, Akamai	10-50ms
Application	Session data, computed results	Redis, Memcached	1-5ms
Database	Query results, indexes	MySQL query cache, pg_bouncer	1-10ms

Caching Strategies

Strategy 1: Cache-Aside (Lazy Loading)

The application manages the cache explicitly.

Loading diagram...

Advantages:

• Only requested data is cached
• Cache failures do not affect application availability
• Compatible with any database

Disadvantages:

• Cache miss requires 3 operations (cache read, DB read, cache write)
• Data can become stale
• Cold start requires cache warming

Use case: Read-heavy workloads, expensive computations

Strategy 2: Write-Through

Writes go to both cache and database synchronously.

Loading diagram...

Advantages:

• Cache is always consistent with database
• Reads are always fast (data is pre-cached)

Disadvantages:

• Increased write latency (dual write)
• Cache may store unread data
• Complex failure handling required

Use case: Data that is read immediately after writing

Strategy 3: Write-Behind (Write-Back)

Writes go to cache immediately; database writes are asynchronous.

Loading diagram...

Advantages:

• Fast write performance
• Batching reduces database load
• Absorbs traffic spikes

Disadvantages:

• Data loss risk if cache fails before database write
• Eventual consistency only
• Complex failure recovery

Use case: Write-heavy workloads, analytics, logging

Strategy 4: Refresh-Ahead

Cache is proactively refreshed before expiration.

Loading diagram...

Advantages:

• Eliminates cache miss latency for frequently accessed data
• Predictable performance

Disadvantages:

• Wasted refreshes for unused data
• Implementation complexity

Use case: Predictable access patterns, critical high-traffic data

Strategy Comparison

Strategy	Write Speed	Read Speed	Consistency	Complexity	Use Case
Cache-Aside	Normal	Miss is slow	Eventual	Low	General purpose
Write-Through	Slower	Always fast	Strong	Medium	Read-after-write
Write-Behind	Fastest	Always fast	Eventual	High	Write-heavy
Refresh-Ahead	Normal	Always fast	Eventual	High	Predictable hot data

Cache Invalidation

Invalidation Strategies

Strategy	Mechanism	Advantages	Disadvantages
TTL (Time-to-Live)	Data expires after fixed time	Simple, automatic	May serve stale data
Event-driven	Invalidate on write/update	Fresh data	Complex, tight coupling
Version-based	Include version in cache key	No invalidation required	Storage overhead
Purge on write	Delete cache entry on update	Simple	Cache miss after every write

TTL Guidelines

Data Type	Suggested TTL	Rationale
Static assets	1 year	Versioned URLs handle updates
User sessions	15-30 minutes	Security requirements
API responses	1-5 minutes	Balance freshness and load
Database query cache	10-60 seconds	Depends on write frequency
Real-time data	No cache or seconds	Freshness critical

Cache Eviction Policies

When cache is full, entries must be removed based on a policy.

Loading diagram...

Policy	Mechanism	Best For	Drawback
LRU	Evict least recently accessed	General workloads	One-time scans pollute cache
LFU	Evict least frequently accessed	Stable hot set	Slow to adapt to changes
FIFO	Evict oldest entries	Simple use cases	Ignores access patterns
TTL	Evict expired entries	Time-sensitive data	May keep stale unused data

LRU is the recommended default for most applications.

Distributed Caching

Single Node vs Distributed

Loading diagram...

Aspect	Local Cache	Distributed Cache
Latency	~1us	~1ms
Consistency	Per-server	Global
Memory	Limited to one server	Scalable
Failure impact	Graceful degradation	Potential single point of failure

Distributed Cache Architecture

Loading diagram...

Cache Technologies

Technology	Type	Strengths	Use Cases
Redis	In-memory, persistent	Data structures, pub/sub, Lua scripts	Sessions, leaderboards, queues
Memcached	In-memory only	Simple, multi-threaded	Simple key-value caching
Varnish	HTTP cache	HTTP-specific optimizations	Web page caching
CDN	Edge cache	Global distribution	Static assets, media

Common Problems

1. Cache Stampede

Multiple requests hit the database simultaneously when cache expires.

Loading diagram...

Solutions:

• Locking: First request acquires lock; others wait
• Probabilistic refresh: Refresh before expiry with randomization
• Background refresh: Separate process maintains cache

2. Hot Key Problem

Single key receives disproportionate traffic.

Solutions:

• Replicate hot keys across multiple nodes
• Add random suffix to distribute load: hot_key_1, hot_key_2
• Use local cache for extremely hot keys

3. Cache Penetration

Requests for non-existent data always reach the database.

Solutions:

• Cache null results with short TTL
• Use Bloom filter to check existence before database query