HTTP Requests (requests)

Python's standard library ships urllib for making HTTP calls, but almost nobody uses it directly — the API is verbose and easy to get wrong. The de facto standard for HTTP in Python is the third-party requests library. It has a small, human-friendly API, handles connection pooling and encoding for you, and is a dependency of countless other packages. If you are talking to a web server or an API from Python, you are almost certainly reaching for requests.

Installing requests

requests is not part of the standard library, so it needs to be installed with pip before you can import it.

Install

Bash

pip install requests

Making your first GET request

The two workhorse functions are requests.get() for reading data and requests.post() for sending data. Both return a Response object that carries the status code, headers, and body.

basic_get.py

Python

import requests

response = requests.get("https://api.github.com")

print(response.status_code)   # 200
print(response.ok)            # True (status_code < 400)
print(response.headers["content-type"])

Checking whether it worked

Every response carries a .status_code. .ok is a convenience boolean that is True for any status code under 400. Never assume a request succeeded just because it did not raise an exception — requests does not raise on a 404 or 500 by default.

Status range	Meaning	Typical cause
2xx	Success	Request was received and handled correctly
3xx	Redirect	Resource moved; requests follows these automatically
4xx	Client error	Bad request, missing auth, wrong URL
5xx	Server error	Something broke on the server

Parsing JSON responses

Most modern APIs return JSON. Response.json() parses the body into native Python data structures (dicts and lists) in one call — no need to import json and call loads() yourself.

parse_json.py

Python

import requests

response = requests.get("https://api.github.com/repos/python/cpython")
data = response.json()

print(data["full_name"])       # "python/cpython"
print(data["stargazers_count"])

Query parameters and headers

Instead of hand-building a query string, pass a dictionary to params= and let requests handle URL-encoding. Custom headers — for example an Authorization header — go in headers=.

params_and_headers.py

Python

import requests

response = requests.get(
    "https://api.github.com/search/repositories",
    params={"q": "language:python", "sort": "stars"},
    headers={"Accept": "application/vnd.github+json"},
    timeout=5,
)

for repo in response.json()["items"][:3]:
    print(repo["full_name"])

Sending data with POST

requests.post() accepts data= for form-encoded bodies and json= for a JSON body (it serializes the dict and sets the Content-Type header for you).

post_json.py

Python

import requests

response = requests.post(
    "https://httpbin.org/post",
    json={"username": "ada", "role": "engineer"},
    timeout=5,
)

print(response.status_code)
print(response.json()["json"])   # echoes back what we sent

Always set a timeout

Requests without a timeout can hang forever

By default, `requests` will wait indefinitely for a server to respond. If the server never responds — a dropped connection, a firewall silently swallowing packets — your program hangs forever with no error. Always pass `timeout=` (in seconds) to every call. `timeout=5` means <= 5 seconds to connect and <= 5 seconds to read the response before a `Timeout` exception is raised.

timeout.py

Python

import requests

# fails fast instead of hanging indefinitely
response = requests.get("https://example.com", timeout=5)

# separate connect/read timeouts if you need finer control
response = requests.get("https://example.com", timeout=(3, 10))

Error handling

requests distinguishes between two kinds of failure: the request never completed (network error, timeout, DNS failure — these raise exceptions automatically), and the request completed but the server returned an error status (404, 500 — these do not raise automatically, you opt in with .raise_for_status()).

error_handling.py

Python

import requests

try:
    response = requests.get(
        "https://api.github.com/repos/does/not-exist",
        timeout=5,
    )
    response.raise_for_status()   # raises HTTPError on 4xx/5xx
    data = response.json()
except requests.exceptions.Timeout:
    print("The request timed out")
except requests.exceptions.HTTPError as exc:
    print(f"Server returned an error: {exc}")
except requests.exceptions.RequestException as exc:
    # catches connection errors, too-many-redirects, and anything
    # else requests can raise
    print(f"Request failed: {exc}")
else:
    print(data)

Catch RequestException as your safety net

`requests.exceptions.RequestException` is the base class for every exception `requests` can raise. Catching it last means you handle specific cases first (timeouts, HTTP errors) and still have a fallback for anything unexpected.

requests.get vs requests.post at a glance

requests.get(url, params=..., headers=..., timeout=...) — read data, parameters go in the URL.
requests.post(url, json=..., data=..., headers=..., timeout=...) — send data, body carries the payload.
response.status_code / response.ok — did it succeed?
response.json() — parse a JSON body into Python objects.
response.raise_for_status() — turn a bad status code into an exception you can catch.

Session objects for repeated calls

If you are making many requests to the same host (e.g. paginating through an API), use `requests.Session()`. It reuses the underlying TCP connection and lets you set headers once instead of on every call.