Mastering Python Requests for HTTP Communication
The requests library provides a high-level interface for handling HTTP requests in Python. Understanding its parameter structure and session handling is essential for effective API interaction.
Parameter Configuration in Requests
The requests.get() and requests.post() methods accept several keyword arguments to modify the outgoing request:
params: Used to pass query string parameters (typically for GET).data: Used for sending form-encoded data (typically for POST).json: Specifically for sending JSON-encoded data in the request body.headers: A dictionary of HTTP headers.timeout: The number of seconds too wait for the server to send data before giving up.files: Used for multipart-encoded file uploads.
Persistence with Session Objects
Standard requests.get() calls are stateless. To maintain cookies across multiple requests, utilize the Session object.
import requests
# Without a session, cookies are not persisted
requests.get('https://httpbin.org/cookies/set/session_id/12345')
lookup = requests.get('https://httpbin.org/cookies')
print(lookup.json()) # Output: {'cookies': {}}
# Using a session object to maintain state
client = requests.Session()
client.get('https://httpbin.org/cookies/set/session_id/12345')
lookup_persistent = client.get('https://httpbin.org/cookies')
print(lookup_persistent.json()) # Output: {'cookies': {'session_id': '12345'}}
SSL Verification and Proxies
When interacting with servers using self-signed certificates, you can bypass SSL verification by setting verify=False.
# Disabling SSL certificate validation
secure_resp = requests.get('https://self-signed.badssl.com/', verify=False)
To route traffic through a proxy, provide a proxies dictionary:
network_gateways = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
api_call = requests.get('https://httpbin.org/ip', proxies=network_gateways)
Authentication Strategies
The library supports various authentication schemes natively.
from requests.auth import HTTPBasicAuth, HTTPDigestAuth
# Basic Authentication
basic_entry = requests.get(
'https://httpbin.org/basic-auth/user/pass',
auth=HTTPBasicAuth('user', 'pass')
)
# Digest Authentication
digest_entry = requests.get(
'https://httpbin.org/digest-auth/auth/user/pass',
auth=HTTPDigestAuth('user', 'pass')
)
Handling File Uploads
Uploading files is handled via the files parameter using a dictionary where the key is the field name and the value is the file-like object.
with open('profile_picture.png', 'rb') as image_data:
upload_payload = {'profile_img': image_data}
upload_response = requests.post('https://httpbin.org/post', files=upload_payload)
print(upload_response.status_code)
URL Encoding
For manual construction of URLs with query strings, urllib.parse.urlencode converts a dictionary into a URL-safe string.
from urllib.parse import urlencode
query_params = {'search': 'python requests', 'lang': 'en'}
base_url = "https://example.com/search?"
final_url = base_url + urlencode(query_params)
Automated Testing Integration
Combining requests with unittest and a report generator allows for robust API testing pipelines.
import unittest
import time
from BSTestRunner import BSTestRunner
def run_api_test_suite():
test_loader = unittest.TestLoader()
suite = test_loader.discover(start_dir='./api_tests', pattern='test_*.py')
timestamp = time.strftime('%Y%H%M%S')
output_file = f'./reports/test_results_{timestamp}.html'
with open(output_file, 'wb') as report_stream:
test_runner = BSTestRunner(
stream=report_stream,
title='System API Regression Report',
description='Automated verification of core endpoints'
)
test_runner.run(suite)