Caching

Caching stores frequently accessed data in faster storage layers to reduce latency and database load. This section covers caching layers, strategies, invalidation approaches, and common problems.

Benefits of Caching

Benefit	Description
Speed	Memory access is approximately 100x faster than disk. A Redis lookup takes microseconds; a database query takes milliseconds.
Protection	Caching absorbs read load, reducing database query volume.
Cost	Reduced compute requirements lower infrastructure costs.

Caching Layers

Loading diagram...

1. Client-Side Cache

Browsers cache content based on HTTP headers. Use Cache-Control to specify caching duration (max-age in seconds) and ETag for conditional requests to validate cached content.

2. CDN (Content Delivery Network)

CDNs cache content at edge locations geographically closer to users.

Use Case	Examples
Static assets	Images, CSS, JavaScript
Media	Video streaming
Pre-rendered content	Static HTML pages

3. Application Cache

In-memory caches like Redis and Memcached store frequently accessed data between the application and database.

Use Case	Examples
Session data	User sessions, authentication tokens
Query results	Database query responses
Computed values	Aggregations, calculations
Rate limiting	Request counters

4. Database Cache

Most databases maintain internal query result caches. These are typically managed automatically by the database engine.

Caching Strategies

Cache-Aside (Lazy Loading)

The application checks the cache first, and only queries the database on a cache miss.

Implementation approach:

1. Check cache using a consistent key format (such as "user:123")
2. If found, return cached value immediately
3. If not found (cache miss), query the database
4. Store the result in cache with an appropriate TTL before returning

Aspect	Description
Advantages	Only caches requested data, handles cache failures gracefully
Disadvantages	Cache miss penalty on first access, potential for stale data

Write-Through

Data is written to both the cache and database simultaneously. On every update, write to the database first, then update the cache with the same data. This ensures cache consistency with the database.

Aspect	Description
Advantages	Cache remains consistent with database
Disadvantages	Increased write latency, caches data that may never be read

Write-Behind (Write-Back)

Data is written to the cache immediately and persisted to the database asynchronously.

Aspect	Description
Advantages	Fast write operations
Disadvantages	Risk of data loss if cache fails before persistence, increased complexity

Cache Invalidation

Cache invalidation determines when cached data should be removed or updated.

Strategies

Strategy	Description	Use Case
TTL (Time To Live)	Data expires after a specified duration	Most common approach
Event-based	Cache is invalidated when underlying data changes	Real-time consistency requirements
Version-based	Version identifier included in cache key	Atomic updates

TTL Configuration

Set TTL (Time To Live) when storing cache entries. For example, a 3600 second TTL expires the entry after 1 hour.

TTL selection factors:

Factor	Consideration
Staleness tolerance	How outdated can data be before it impacts users?
Read/write ratio	Higher read ratios benefit more from longer TTLs
Cache miss cost	Expensive operations warrant longer TTLs

Cache Problems

Cache Stampede

Multiple requests simultaneously miss the cache and query the database, causing load spikes.

Solution	Description
Staggered TTLs	Add random variation to expiration times
Lock/semaphore	Only one request populates the cache; others wait
Background refresh	Refresh cache before expiration

Hot Keys

A single cache key receives disproportionately high traffic.

Solution	Description
Key replication	Distribute across multiple cache keys
Local caching	Cache in application memory
Rate limiting	Limit requests to hot keys

Cache Penetration

Requests for non-existent data bypass the cache and query the database repeatedly.

Solution	Description
Cache negative results	Store "not found" responses
Bloom filter	Pre-filter requests for non-existent keys
Rate limiting	Limit requests for unknown keys

Redis vs Memcached

Feature	Redis	Memcached
Data structures	Rich (lists, sets, sorted sets)	Key-value only
Persistence	Yes	No
Replication	Yes	No
Clustering	Yes	Client-side
Memory efficiency	Lower	Higher

Selection criteria:

Use Case	Recommendation
Complex data structures (sorted sets, pub/sub)	Redis
Data persistence required	Redis
Simple key-value cache with maximum memory efficiency	Memcached

Design Considerations

Consideration	Description
Read/write ratio	Caching provides greatest benefit for read-heavy workloads
Staleness tolerance	Determines cache strategy and TTL settings
Failure handling	System should degrade gracefully when cache is unavailable
Cache warming	Plan for populating cache on cold starts
Capacity planning	Cache must accommodate working set of frequently accessed data