3.16.a — What is Caching

Caching is the practice of storing copies of frequently accessed data in a faster storage layer so that future requests for that data can be served more quickly, reducing database load and improving response times.

---

<< README | Next: 3.16.b — Introduction to Redis >>

---

1. Why Cache?

Every time your API handles a request that needs data from a database, it performs a query. These queries take time -- network latency to the database, query execution, data serialization. For data that does not change often, repeating this work on every request is wasteful.

WITHOUT CACHING (every request hits the database):
  Request → Express → MongoDB Query (50ms) → Response
  Request → Express → MongoDB Query (50ms) → Response
  Request → Express → MongoDB Query (50ms) → Response
  ↑ 50ms per request, database handles every request

WITH CACHING (most requests served from memory):
  Request → Express → Cache HIT (0.5ms) → Response
  Request → Express → Cache HIT (0.5ms) → Response
  Request → Express → Cache MISS → MongoDB (50ms) → Store in Cache → Response
  ↑ 0.5ms for cache hits (100x faster), database handles far fewer requests

Benefits of Caching

Benefit	Impact
Faster responses	Cache reads are ~100-1000x faster than database queries
Reduced database load	Fewer queries means database handles more users
Lower infrastructure costs	Less database scaling needed
Better user experience	Sub-millisecond responses feel instant
Improved availability	Cache can serve during database maintenance

---

2. Cache Hit vs Cache Miss

┌──────────┐     ┌───────────┐
│  Request  │────>│   Cache   │
└──────────┘     └─────┬─────┘
                       │
              ┌────────┴────────┐
              │                 │
         CACHE HIT         CACHE MISS
         (data found)      (data not found)
              │                 │
              ▼                 ▼
         Return data     ┌──────────┐
         immediately     │ Database │
                         └────┬─────┘
                              │
                         Store in cache
                         for next time
                              │
                         Return data

• Cache Hit: Data found in cache. Return immediately. Fast.
• Cache Miss: Data not in cache. Query database, store result in cache, return data. Slow (first time only).
• Hit Ratio: Percentage of requests served from cache. Aim for 80-95%+.

Hit Ratio = (cache hits) / (cache hits + cache misses) * 100

Example: 950 hits, 50 misses → 950/1000 * 100 = 95% hit ratio

---

3. Caching Patterns

Cache-Aside (Lazy Loading)

The most common pattern. The application manages the cache explicitly.

async function getProduct(productId) {
  // 1. Check the cache first
  const cached = await cache.get(`product:${productId}`);
  if (cached) {
    return JSON.parse(cached); // CACHE HIT
  }

  // 2. Cache miss — query the database
  const product = await Product.findById(productId);

  // 3. Store in cache for next time (TTL: 1 hour)
  await cache.set(`product:${productId}`, JSON.stringify(product), { EX: 3600 });

  // 4. Return the data
  return product;
}

Pros: Only caches data that is actually requested. Cache failures do not break the app. Cons: First request is always slow (cache miss). Cache can become stale.

Write-Through

Every write goes to the cache first, which then writes to the database. Cache and database are always in sync.

async function updateProduct(productId, updates) {
  // 1. Update the database
  const product = await Product.findByIdAndUpdate(productId, updates, { new: true });

  // 2. Update the cache immediately
  await cache.set(`product:${productId}`, JSON.stringify(product), { EX: 3600 });

  return product;
}

Pros: Cache is always up to date. No stale data. Cons: Write latency increases (must write to both). Caches data that may never be read.

Write-Behind (Write-Back)

Write to cache immediately, then asynchronously write to the database later.

async function updateProduct(productId, updates) {
  // 1. Update the cache immediately (fast)
  await cache.set(`product:${productId}`, JSON.stringify(updates));

  // 2. Queue the database write for later (async)
  writeQueue.add({ collection: 'products', id: productId, updates });

  return updates; // Return immediately, DB write happens later
}

Pros: Very fast writes. Batch database writes for efficiency. Cons: Risk of data loss if cache fails before DB write. Complex implementation.

---

4. Cache Invalidation: The Hardest Problem

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

Cache invalidation means removing or updating cached data when the source data changes.

Strategies

Strategy	How it Works	Use Case
TTL (Time To Live)	Data expires automatically after a set time	Most common, good default
Event-based	Invalidate cache when data changes	When you control all write paths
Version-based	Include a version number in cache keys	When data has clear versions
Manual	Explicitly delete cache entries	Admin operations, migrations

TTL-Based Invalidation

// Data expires after 1 hour — automatically removed
await cache.set('popular-posts', JSON.stringify(posts), { EX: 3600 });

// After 3600 seconds, the next request will be a cache miss,
// causing a fresh database query and a new cache entry.

Event-Based Invalidation

// When a product is updated, invalidate its cache
async function updateProduct(productId, updates) {
  const product = await Product.findByIdAndUpdate(productId, updates, { new: true });

  // Invalidate the specific cache entry
  await cache.del(`product:${productId}`);

  // Also invalidate any lists that include this product
  await cache.del('products:all');
  await cache.del(`products:category:${product.category}`);

  return product;
}

---

5. TTL (Time To Live)

TTL determines how long cached data remains valid before automatic expiration.

// Set TTL when caching
await cache.set('key', 'value', { EX: 60 });     // Expires in 60 seconds
await cache.set('key', 'value', { EX: 3600 });   // Expires in 1 hour
await cache.set('key', 'value', { EX: 86400 });  // Expires in 24 hours

// Check remaining TTL
const remaining = await cache.ttl('key'); // Returns seconds remaining
// -1 = no expiration set
// -2 = key does not exist

Choosing TTL Values

Data Type	Suggested TTL	Reasoning
User session	24 hours	Balance security and convenience
Product catalog	1-6 hours	Changes infrequently
Search results	5-15 minutes	Changes moderately
API rate limit counters	1 minute	Must be fresh
Static config	24 hours	Rarely changes
Real-time stock prices	5-30 seconds	Changes constantly
User profile	30-60 minutes	Changes occasionally

---

6. Local Caching: In-Memory Solutions

Before adding Redis, you can use simple in-memory caching for single-process applications.

Using a Plain JavaScript Map

// Simple in-memory cache with TTL
class SimpleCache {
  constructor() {
    this.store = new Map();
  }

  set(key, value, ttlSeconds) {
    const expiry = ttlSeconds ? Date.now() + (ttlSeconds * 1000) : null;
    this.store.set(key, { value, expiry });
  }

  get(key) {
    const entry = this.store.get(key);
    if (!entry) return null;

    // Check if expired
    if (entry.expiry && Date.now() > entry.expiry) {
      this.store.delete(key);
      return null;
    }

    return entry.value;
  }

  del(key) {
    this.store.delete(key);
  }

  clear() {
    this.store.clear();
  }
}

const cache = new SimpleCache();
cache.set('greeting', 'Hello', 60); // Expires in 60s
console.log(cache.get('greeting')); // "Hello"

Using node-cache Package

const NodeCache = require('node-cache');

// stdTTL: default TTL in seconds, checkperiod: cleanup interval
const cache = new NodeCache({ stdTTL: 600, checkperiod: 120 });

// Set with default TTL (600s)
cache.set('user:123', { name: 'Alice', email: 'alice@example.com' });

// Set with custom TTL
cache.set('temp-data', 'value', 30); // 30 seconds

// Get
const user = cache.get('user:123'); // Returns the object or undefined

// Delete
cache.del('user:123');

// Events
cache.on('expired', (key, value) => {
  console.log(`Key expired: ${key}`);
});

Limitations of Local Caching

Limitation	Explanation
Single process only	Each Node.js process has its own cache. No sharing between cluster workers or multiple servers.
Lost on restart	Memory is cleared when the process restarts.
Memory limits	Limited by the process's available memory. No eviction policies.
No persistence	Data exists only in RAM.

This is why Redis exists -- it solves all of these problems.

---

7. When to Cache (and When NOT to)

Good Candidates for Caching

What to Cache	Why
Database query results (read-heavy)	Expensive queries, rarely changing data
API responses from third-party services	Rate limits, slow external calls
Computed/aggregated data	Dashboard statistics, report summaries
Session data	Accessed on every request
Static configuration	Feature flags, app settings
Frequently accessed reference data	Country lists, category trees

Bad Candidates for Caching

What NOT to Cache	Why
Rapidly changing data	Cache would be stale almost immediately
Sensitive personal data (without encryption)	Security risk if cache is compromised
Write-heavy data	Cache invalidation on every write negates the benefit
Large binary files	Better served via CDN
Data that must be perfectly real-time	Any staleness is unacceptable (financial transactions)
Data unique to each request	Cache hit ratio would be near 0%

Decision Framework

Should I cache this data?
├── Is it read frequently? (>10 reads per write)
│   ├── Yes → Strong candidate for caching
│   └── No → Probably not worth caching
├── Is it expensive to generate? (>50ms query time)
│   ├── Yes → Strong candidate
│   └── No → Low priority
├── Can you tolerate stale data for a short time?
│   ├── Yes → Cache with appropriate TTL
│   └── No → Cache with event-based invalidation or skip
└── Is the data the same for many users?
    ├── Yes → Very efficient to cache (high hit ratio)
    └── No → Per-user caching (lower hit ratio, still valuable)

---

8. Cache Stampede Problem

When a popular cache entry expires, many simultaneous requests all experience a cache miss and all query the database at the same time, potentially overwhelming it.

Cache expires for "popular-posts"
  ↓
Request 1 → Cache MISS → Query DB ─┐
Request 2 → Cache MISS → Query DB ─┤ All hit the database
Request 3 → Cache MISS → Query DB ─┤ at the exact same time!
Request 4 → Cache MISS → Query DB ─┘

Solutions

// Solution 1: Mutex/Lock — only one request queries the DB
const locks = new Map();

async function getWithLock(key, fetchFn, ttl) {
  const cached = await cache.get(key);
  if (cached) return JSON.parse(cached);

  // Check if another request is already fetching
  if (locks.has(key)) {
    // Wait for the other request to finish
    return locks.get(key);
  }

  // Acquire lock
  const promise = fetchFn().then(async (data) => {
    await cache.set(key, JSON.stringify(data), { EX: ttl });
    locks.delete(key);
    return data;
  });

  locks.set(key, promise);
  return promise;
}

// Solution 2: Stale-While-Revalidate — serve stale data while refreshing
async function getWithRevalidate(key, fetchFn, ttl) {
  const cached = await cache.get(key);
  const staleCached = await cache.get(`stale:${key}`);

  if (cached) return JSON.parse(cached);

  if (staleCached) {
    // Serve stale data immediately, refresh in background
    refreshInBackground(key, fetchFn, ttl);
    return JSON.parse(staleCached);
  }

  // No data at all — must wait for DB
  const data = await fetchFn();
  await cache.set(key, JSON.stringify(data), { EX: ttl });
  await cache.set(`stale:${key}`, JSON.stringify(data), { EX: ttl * 2 });
  return data;
}

---

Key Takeaways

1. Caching stores data in fast storage (memory) to avoid repeated expensive operations
2. Cache-aside is the most common pattern: check cache, miss triggers DB query, store result
3. Cache invalidation is the hardest problem -- use TTL as a safety net, event-based for precision
4. TTL should match how frequently your data changes and how much staleness you can tolerate
5. Local caching (Map, node-cache) works for single-process apps but does not scale
6. Redis solves the limitations of local caching: shared, persistent, rich data structures
7. Not everything should be cached -- focus on read-heavy, expensive-to-compute, infrequently changing data
8. Watch out for cache stampede on high-traffic entries

---

Explain-It Challenge

Scenario: Your e-commerce API has an endpoint GET /api/products?category=electronics&sort=popular that takes 800ms because it joins 3 database tables, filters, sorts, and paginates. This endpoint is called 500 times per minute. Products are updated by sellers about 20 times per hour.

Design a caching strategy. What would you use as the cache key? What TTL would you set? How would you handle cache invalidation when a seller updates their product? What about pagination -- do you cache each page separately? Calculate the potential improvement in database load.

---

<< README | Next: 3.16.b — Introduction to Redis >>