Symptom

Calling requests.get against an HTTP URL appears to work once, but a subsequant call raises an SSL error similar to:

requests.exceptions.SSLError: HTTPSConnectionPool(host='www.baidu.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)')))

Reproducing the issue

Example target: http://www.baidu.com/

1. No headers supplied (HTTP stays HTTP):

import requests

resp = requests.get('http://www.baidu.com/', timeout=5)
print(resp.status_code, resp.url)          # e.g., 200, 'http://www.baidu.com/'

2. Spoofing a desktop browser User-Agent (server may switch to HTTPS):

import requests

ua = {
    'User-Agent': (
        'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1'
    )
}

# First request
r1 = requests.get('http://www.baidu.com/', headers=ua, timeout=5)
print('first:', r1.status_code, r1.url)

# A follow-up request may fail if HTTPS validation cannot be performed
try:
    r2 = requests.get('http://www.baidu.com/', headers=ua, timeout=5)
    print('second:', r2.status_code, r2.url)
except requests.exceptions.SSLError as exc:
    print('second raised:', type(exc).__name__, exc)

Why this happens

Some sites decide whether to redirect http → https based on the User-Agent. When redirected to https://www.baidu.com/, requests will validate the TLS certificate by default. If your Python/requests/SSL trust store is outdated or misconfigured, certificate verification fails and triggers SSLError via HTTPSConnectionPool.

Confirming the redirect behavior

Disable certificate checking temporarily to inspect redirect history. This is only for diagnosis.

import requests

ua = {
    'User-Agent': (
        'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1'
    )
}

# Case A: UA set — expect http -> https redirect
with requests.Session() as s:
    resp = s.get('http://www.baidu.com/', headers=ua, verify=False, timeout=5)
    print([h.status_code for h in resp.history])   # e.g., [302]
    print(resp.url)                                 # 'https://www.baidu.com/'

# Case B: No UA — often stays on http
resp2 = requests.get('http://www.baidu.com/', verify=False, timeout=5)
print([h.status_code for h in resp2.history])       # e.g., []
print(resp2.url)                                    # 'http://www.baidu.com/'

Note: You will see InsecureRequestWarning when verify=False; this is expected during diagnosis.

Practical fixes

Prefer fixing trust rather than disabling verification.

• Use a current CA bundle (certifi) and point requests to it:

import requests, certifi

# Directly use HTTPS and a modern CA bundle
r = requests.get('https://www.baidu.com/', verify=certifi.where(), timeout=5)
print(r.status_code)

• Update your toolchain:

  • Upgrade requests and urllib3
  • Ensure Python’s OpenSSL is up-to-date
  • On Windows, install/upgrade certifi; on macOS/Linux, ensure system CA store is current

• Configure CA paths explicitly if needed:

  • Environment variables: REQUESTS_CA_BUNDLE or SSL_CERT_FILE
  • Or supply a PEM bundle path via verify='/path/to/ca-bundle.pem'

• As a last resort for non-production scenarios, disable verification:

import requests

r = requests.get('https://www.baidu.com/', verify=False, timeout=5)
print(r.status_code)

To examine redirects without triggering verification errors, you can also avoid following redirects automatically and inspect the Location header:

import requests

ua = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1'
}

r = requests.get('http://www.baidu.com/', headers=ua, allow_redirects=False, timeout=5)
print(r.status_code)           # 301/302 when redirecting
print(r.headers.get('Location'))

Useful references

• Browser User-Agent strings: https://www.useragentstring.com/
• Requests SSL/TLS verification: https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification