Health Check Patterns

TL;DR

Explanation

Kubernetes probes: liveness (restart if failing), readiness (remove from load balancer if failing), startup (wait for slow start before liveness kicks in). Patterns: shallow (is process alive? just return 200), deep (check DB, cache, queue), readiness (are all dependencies ready?). Shallow for liveness — avoid cascading failures (if DB is down, all instances fail liveness, all restart, making DB situation worse). Deep for readiness — remove from rotation without restarting. Deep health check components: DB query (SELECT 1), cache ping, queue connectivity, disk space check, memory check. Timeout all checks (1-2s max).

Common Misconception

Why It Matters

Common Mistakes

• Deep health check for liveness — DB down → all instances restart → thundering herd.
• No readiness probe — instances receive traffic before warming up.
• Health check without timeout — slow dependency causes health check to time out and restart healthy instance.

Code Examples

// Liveness checks DB — cascading failure on DB outage:
route('/health/live', function() {
    DB::select('SELECT 1'); // DB down → liveness fails → instance restarts → more DB pressure
    return ['status' => 'ok'];
});

// Shallow liveness — is process running?
route('/health/live', fn() => ['status' => 'alive']);

// Deep readiness — can serve traffic?
route('/health/ready', function() {
    $checks = [
        'db' => fn() => DB::select('SELECT 1'),
        'cache' => fn() => Cache::has('health-check') || Cache::set('health-check', 1, 10),
    ];
    foreach ($checks as $name => $check) {
        try { $check(); } catch (Exception $e) {
            return response(['status' => 'not ready', 'failed' => $name], 503);
        }
    }
    return ['status' => 'ready'];
});

References

• ↗ https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Tags

Related Terms