HTTP in Python — requests & httpx

Also Known As

TL;DR

Explanation

requests: synchronous HTTP with a clean API — Session for connection pooling, automatic JSON encoding/decoding, timeout parameter (always set it), retry with HTTPAdapter. httpx: requests-compatible API plus async (async with httpx.AsyncClient()), HTTP/2 support, better streaming. Key practices: always set timeout, use sessions for multiple requests to the same host (connection reuse), check response.raise_for_status() rather than manual status code checking, and never disable SSL verification (verify=False is insecure).

Common Misconception

Why It Matters

Common Mistakes

• No timeout parameter — process hangs indefinitely on slow servers.
• Not using Session for multiple requests — creates a new TCP connection per request.
• verify=False to bypass SSL — disables certificate verification, enabling MITM.
• Not calling raise_for_status() — 400/500 responses silently succeed without it.

Code Examples

import requests
# No timeout — hangs forever if server is slow:
response = requests.get('https://api.example.com/data')
# No error check — 500 treated as success:
data = response.json()

# New connection every request:
for url in urls:
    requests.get(url)  # New TCP connection each time

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

# Session with retry and timeout:
session = requests.Session()
retry = Retry(total=3, backoff_factor=0.3, status_forcelist=[500, 502, 503])
session.mount('https://', HTTPAdapter(max_retries=retry))

response = session.get('https://api.example.com/data', timeout=(3.05, 27))
response.raise_for_status()  # Raises for 4xx/5xx
data = response.json()

# Async with httpx:
async with httpx.AsyncClient(timeout=30) as client:
    response = await client.get(url)
    response.raise_for_status()

Tags

Related Terms

References

• https://docs.python-requests.org/
• https://www.python-httpx.org/