Thundering Herd Problem

TL;DR

Explanation

Cache stampede (a thundering herd variant): cache entry expires → hundreds of concurrent requests all miss → all hit the DB simultaneously → DB overloaded → all queries slow/timeout → all requests fail. Solutions: (1) Cache locking (mutex on cache miss — only one regenerates). (2) Cache stale-while-revalidate (serve stale during regeneration). (3) Probabilistic early expiry (randomly regenerate before expiry). (4) Cache warming before expiry. Another variant: all workers wake on a queue event — only one gets the job but all CPU-spin. Fix: use SKIP LOCKED or randomised polling intervals.

Common Misconception

Why It Matters

Common Mistakes

• Long TTL to avoid expiry — just delays the stampede.
• Not locking cache regeneration — every request races to regenerate.
• Simultaneous queue worker start — all workers poll at the same time.

Code Examples

// Cache stampede:
$value = $cache->get('expensive_query');
if ($value === null) {
    $value = $db->runExpensiveQuery(); // All 500 concurrent requests hit this
    $cache->set('expensive_query', $value, 3600);
}

// Mutex on cache miss:
$value = $cache->get('expensive_query');
if ($value === null) {
    $lock = $redis->set('lock:expensive_query', 1, ['NX','EX'=>10]);
    if ($lock) {
        $value = $db->runExpensiveQuery();
        $cache->set('expensive_query', $value, 3600);
        $redis->del('lock:expensive_query');
    } else {
        // Wait briefly then re-check cache:
        usleep(100000);
        $value = $cache->get('expensive_query') ?? $fallbackValue;
    }
}

References

• ↗ https://en.wikipedia.org/wiki/Thundering_herd_problem

Tags

Related Terms