Unable to request a private URL endpoint with custom SNI and self-signed CA when the proxy is set
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
Recently I was using requests_toolbelt alongside the requests library, more specifically, I was leveraging the HostHeaderSSLAdapter from requests_toolbelt to make sure I could request a private URL endpoint (e.g. https://1.2.3.4:5678/api/check) with the custom SNI and self-signed CA certificate.
It works great until I try to request the same thing with a proxy, and it starts to occur the SSL verification exception as follows:
requests.exceptions.SSLError: HTTPSConnectionPool(host='1.2.3.4', port=5678): Max retries exceeded with url: /api/check_token (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: IP address mismatch, certificate is not valid for '1.2.3.4:'. (_ssl.c:1006)")))
I thought this was a bug related to the requests_toolbelt at first, like the existing bug report #276, but after doing some digging, I'm pretty sure it is bound to the requests itself, hence I report the bug here and a PR that for it later.
Expected Result
Everything works the same way with or without a proxy.
Actual Result
- Private URL endpoint + custom SNI + self-signed CA ✅
- Private URL endpoint + custom SNI + self-signed CA + HTTP Proxy ❌
Reproduction Steps
import requests
from requests_toolbelt.adapters.host_header_ssl import HostHeaderSSLAdapter
session = requests.Session()
session.trust_env = False
session.mount('https://', HostHeaderSSLAdapter())
# Modify http://127.0.0.1:20809 to your actual proxy url
session.proxies.update({'https': 'http://127.0.0.1:20809',
'http': 'http://127.0.0.1:20809'})
# Modify https://1.2.3.4:5678/api/check to your actual private URL endpoint
resp = session.get('https://1.2.3.4:5678/api/check',
headers={'Host': '{YOUR_HOST_NAME}'},
verify='{YOUR_SELF_SIGNED_CA_FILE}')
print(resp.status_code, resp.content)
System Information
No need
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the supplied Python reproduction with requests_toolbelt, an HTTP proxy, and the self-signed CA. Trace how the proxy path handles the custom SNI and certificate verification, then confirm that the same request succeeds both with and without the proxy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100