{ "slug": "api_request_timeout_handling", "term": "API Request Timeout Handling", "category": "api_design", "difficulty": "intermediate", "short": "Client-side deadlines, retries with backoff, and circuit breakers that keep your app responsive when an upstream API fails to reply in time.", "long": "Request timeout handling is the client-side discipline of bounding how long you wait for an upstream API and deciding what to do when that bound is exceeded. The single most common production incident with HTTP clients is an unbounded request that hangs forever, exhausting connection pools, threads, or worker processes until the whole service falls over. Every outbound call needs an explicit connection timeout (time to establish the TCP/TLS handshake) and a read/response timeout (time to receive the response body). A total deadline that spans retries is even better, so a slow dependency cannot blow your own SLA.\n\nWhen a timeout fires you have a decision to make. For idempotent operations (GET, PUT, DELETE, or POST guarded by an idempotency key) you can retry, but only with exponential backoff and jitter so that a recovering server is not hammered by a synchronised retry storm. Blind retries on non-idempotent writes can double-charge a customer or duplicate an order. Cap the number of attempts and the total elapsed time, and propagate a remaining-budget deadline so nested calls do not each restart the clock.\n\nA circuit breaker adds a layer above retries: after a threshold of failures it trips open and fails fast for a cooldown window instead of letting every request wait out the full timeout. This protects both the caller (no thread pile-up) and the struggling dependency (no thundering herd). After the cooldown it allows a trial request (half-open) before closing again.\n\nGood timeout handling also means surfacing a meaningful error to the caller (504 Gateway Timeout or a degraded fallback) rather than a generic 500, and emitting metrics so you can see timeout rates climb before they cause an outage. The goal is graceful degradation: a slow dependency should make one feature slow, not take the entire application down.", "aliases": [ "request timeouts", "retry and backoff", "circuit breaker", "client resilience" ], "tags": [ "api-design", "resilience", "retries", "circuit-breaker", "timeouts", "fault-tolerance" ], "misconception": "A retry on failure always makes the call more reliable. Retrying a non-idempotent write after a timeout can duplicate the operation, and retrying without backoff amplifies load on an already-struggling server.", "why_it_matters": "An outbound HTTP call without a timeout will eventually hang and exhaust your connection pool or worker processes, turning one slow dependency into a full outage. Bounded deadlines, backoff, and circuit breakers contain the blast radius.", "common_mistakes": [ "Making HTTP calls with no explicit connection or read timeout, so a stalled upstream hangs the request indefinitely.", "Retrying non-idempotent POST requests after a timeout, causing duplicate charges or duplicate records.", "Retrying immediately without exponential backoff and jitter, creating a synchronised retry storm against a recovering server.", "Restarting the full timeout budget on each nested call so a chain of three calls can take 3x your intended deadline.", "Catching the timeout and returning a generic 500 instead of a 504 or a degraded fallback the caller can act on." ], "when_to_use": [ "Apply explicit connect and read timeouts to every outbound API call, with no exceptions.", "Use exponential backoff with jitter and a capped attempt count whenever you retry idempotent requests.", "Add a circuit breaker for high-traffic dependencies where a slow upstream could exhaust your connection pool or worker threads.", "Propagate a shared remaining-time deadline across a chain of nested service calls so the whole request respects one budget." ], "avoid_when": [ "Do not add automatic retries to non-idempotent writes unless they are protected by an idempotency key or server-side deduplication.", "Avoid aggressive low timeouts on legitimately long operations (large uploads, report generation) - use a higher per-route budget or async processing instead.", "Skip a circuit breaker for a dependency that is never on a hot path and has no fan-out impact - the added complexity is not worth it." ], "related": [ "api_idempotency_keys", "api_rate_limiting_design", "api_error_handling", "p99_latency", "database_connection_pool" ], "prerequisites": [ "api_idempotency_keys", "api_error_handling", "tcp_ip_model" ], "refs": [ "https://learn.microsoft.com/en-us/azure/architecture/patterns/circuit-breaker", "https://learn.microsoft.com/en-us/azure/architecture/patterns/retry", "https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/", "https://www.rfc-editor.org/rfc/rfc7231#section-6.6.5" ], "bad_code": "post('https://payments.example.com/charges', [\n 'json' => $payload,\n ]);\n return json_decode((string) $res->getBody(), true);\n } catch (\\Exception $e) {\n // Immediate retry, no backoff -> retry storm.\n // And we may have already charged the customer once.\n }\n }\n throw new \\RuntimeException('charge failed');\n}", "good_code": " 2.0, // TCP/TLS handshake budget\n 'timeout' => 5.0, // total response budget\n]);\n\nfunction chargeCustomer(Client $client, array $payload, string $idempotencyKey): array\n{\n $maxAttempts = 3;\n $base = 0.2; // seconds\n\n for ($attempt = 1; $attempt <= $maxAttempts; $attempt++) {\n try {\n $res = $client->post('https://payments.example.com/charges', [\n // Idempotency key makes the retry safe: server dedupes.\n 'headers' => ['Idempotency-Key' => $idempotencyKey],\n 'json' => $payload,\n ]);\n return json_decode((string) $res->getBody(), true);\n } catch (ConnectException | ServerException $e) {\n if ($attempt === $maxAttempts) {\n throw $e; // surface as 504 upstream\n }\n // Exponential backoff with full jitter (seconds).\n $ceiling = min($base * (2 ** ($attempt - 1)), 2.0);\n $sleep = mt_rand(0, (int) ($ceiling * 1000)) / 1000.0;\n usleep((int) ($sleep * 1_000_000));\n }\n }\n throw new \\RuntimeException('charge failed');\n}", "quick_fix": "Set explicit connect and read timeouts on every HTTP client, and only retry idempotent calls using exponential backoff with jitter and a capped attempt count.", "severity": "high", "effort": "medium", "created": "2026-06-12", "updated": "2026-06-12", "citation": { "canonical_url": "https://codeclaritylab.com/glossary/api_request_timeout_handling", "html_url": "https://codeclaritylab.com/glossary/api_request_timeout_handling", "json_url": "https://codeclaritylab.com/glossary/api_request_timeout_handling.json", "source": "CodeClarityLab Glossary", "author": "P.F.", "author_url": "https://pfmedia.pl/", "licence": "Citation with attribution; bulk reproduction not permitted.", "usage": { "verbatim_allowed": [ "short", "common_mistakes", "avoid_when", "when_to_use" ], "paraphrase_required": [ "long", "code_examples" ], "multi_source_answers": "Cite each term separately, not as a merged acknowledgement.", "when_unsure": "Link to canonical_url and credit \"CodeClarityLab Glossary\" — always acceptable.", "attribution_examples": { "inline_mention": "According to CodeClarityLab: ", "markdown_link": "[API Request Timeout Handling](https://codeclaritylab.com/glossary/api_request_timeout_handling) (CodeClarityLab)", "footer_credit": "Source: CodeClarityLab Glossary — https://codeclaritylab.com/glossary/api_request_timeout_handling" } } } }