API Request Timeout Handling

Also Known As

TL;DR

Explanation

Request timeout handling is the client-side discipline of bounding how long you wait for an upstream API and deciding what to do when that bound is exceeded. The single most common production incident with HTTP clients is an unbounded request that hangs forever, exhausting connection pools, threads, or worker processes until the whole service falls over. Every outbound call needs an explicit connection timeout (time to establish the TCP/TLS handshake) and a read/response timeout (time to receive the response body). A total deadline that spans retries is even better, so a slow dependency cannot blow your own SLA.

When a timeout fires you have a decision to make. For idempotent operations (GET, PUT, DELETE, or POST guarded by an idempotency key) you can retry, but only with exponential backoff and jitter so that a recovering server is not hammered by a synchronised retry storm. Blind retries on non-idempotent writes can double-charge a customer or duplicate an order. Cap the number of attempts and the total elapsed time, and propagate a remaining-budget deadline so nested calls do not each restart the clock.

A circuit breaker adds a layer above retries: after a threshold of failures it trips open and fails fast for a cooldown window instead of letting every request wait out the full timeout. This protects both the caller (no thread pile-up) and the struggling dependency (no thundering herd). After the cooldown it allows a trial request (half-open) before closing again.

Good timeout handling also means surfacing a meaningful error to the caller (504 Gateway Timeout or a degraded fallback) rather than a generic 500, and emitting metrics so you can see timeout rates climb before they cause an outage. The goal is graceful degradation: a slow dependency should make one feature slow, not take the entire application down.

Common Misconception

Why It Matters

Common Mistakes

• Making HTTP calls with no explicit connection or read timeout, so a stalled upstream hangs the request indefinitely.
• Retrying non-idempotent POST requests after a timeout, causing duplicate charges or duplicate records.
• Retrying immediately without exponential backoff and jitter, creating a synchronised retry storm against a recovering server.
• Restarting the full timeout budget on each nested call so a chain of three calls can take 3x your intended deadline.
• Catching the timeout and returning a generic 500 instead of a 504 or a degraded fallback the caller can act on.

Avoid When

• Do not add automatic retries to non-idempotent writes unless they are protected by an idempotency key or server-side deduplication.
• Avoid aggressive low timeouts on legitimately long operations (large uploads, report generation) - use a higher per-route budget or async processing instead.
• Skip a circuit breaker for a dependency that is never on a hot path and has no fan-out impact - the added complexity is not worth it.

When To Use

• Apply explicit connect and read timeouts to every outbound API call, with no exceptions.
• Use exponential backoff with jitter and a capped attempt count whenever you retry idempotent requests.
• Add a circuit breaker for high-traffic dependencies where a slow upstream could exhaust your connection pool or worker threads.
• Propagate a shared remaining-time deadline across a chain of nested service calls so the whole request respects one budget.

Code Examples

<?php
use GuzzleHttp\Client;

// No timeouts: this call can hang forever and pin a worker.
$client = new Client();

function chargeCustomer(Client $client, array $payload): array
{
    // Naive retry loop on a non-idempotent POST.
    for ($i = 0; $i < 3; $i++) {
        try {
            $res = $client->post('https://payments.example.com/charges', [
                'json' => $payload,
            ]);
            return json_decode((string) $res->getBody(), true);
        } catch (\Exception $e) {
            // Immediate retry, no backoff -> retry storm.
            // And we may have already charged the customer once.
        }
    }
    throw new \RuntimeException('charge failed');
}

<?php
use GuzzleHttp\Client;
use GuzzleHttp\Exception\ConnectException;
use GuzzleHttp\Exception\ServerException;

$client = new Client([
    'connect_timeout' => 2.0,  // TCP/TLS handshake budget
    'timeout'         => 5.0,  // total response budget
]);

function chargeCustomer(Client $client, array $payload, string $idempotencyKey): array
{
    $maxAttempts = 3;
    $base = 0.2; // seconds

    for ($attempt = 1; $attempt <= $maxAttempts; $attempt++) {
        try {
            $res = $client->post('https://payments.example.com/charges', [
                // Idempotency key makes the retry safe: server dedupes.
                'headers' => ['Idempotency-Key' => $idempotencyKey],
                'json'    => $payload,
            ]);
            return json_decode((string) $res->getBody(), true);
        } catch (ConnectException | ServerException $e) {
            if ($attempt === $maxAttempts) {
                throw $e; // surface as 504 upstream
            }
            // Exponential backoff with full jitter (seconds).
            $ceiling = min($base * (2 ** ($attempt - 1)), 2.0);
            $sleep = mt_rand(0, (int) ($ceiling * 1000)) / 1000.0;
            usleep((int) ($sleep * 1_000_000));
        }
    }
    throw new \RuntimeException('charge failed');
}

Tags

Related Terms

References

• https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker
• https://learn.microsoft.com/en-us/azure/architecture/patterns/retry
• https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
• https://www.rfc-editor.org/rfc/rfc7231#section-6.6.5