urllib.request.proxy_bypass makes DNS request without using configured proxies when system has NO_PROXY configured at some level
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 54.3k
- Forks
- 10.4k
- Avg merge
- 16h 43m
- Merged PRs (30d)
- 3
Description
When using Requests via a SOCKS5 proxy on one of the latest MacBook Pros (Apple Silicon), the DNS requests are not correctly passed to the proxy server even when using the "socks5h" scheme. Instead, the DNS resolution happens at the level of the client, causing a DNS leak.
This happens with various versions/combinations of Requests/PySocks/Python, the latter being the Apple Silicon versions obtained through pyenv. I have tested this using the local SOCKS5 proxy server provided by Tor.
In more detail, I tested the bug on various pyenv Python distributions, labeled 3.8 to 3.10 plus miniforge3, with Requests from v2.16.0 to v2.27.1, and PySocks from v1.5.7 to v1.7.1 (not every single version in between). In what follows, I'll use an example System Information output.
Also, I used Wireshark to monitor the DNS requests, and I used curl with the --proxy socks5h://127.0.0.1:9050 flag as a control, to make sure that the local Tor proxy was working properly and that the DNS leak is indeed specific to Python/Requests/PySocks.
I am not able to confirm whether the DNS leak also happens on Python versions obtained other than through pyenv. On an Intel MacBook using an Anaconda Python version and the latest Requests/PySocks, the leak does not occur.
Expected Result
When using the "socks5h" scheme, the DNS requests should be forwarded to the SOCKS proxy.
Actual Result
The DNS requests are sent from the client instead, causing a DNS leak.
Reproduction Steps
Install any Apple Silicon Python version from pyenv (see above for the versions I tested) and start Tor.
import requests
proxies = {
"http": "socks5h://127.0.0.1:9050",
"https": "socks5h://127.0.0.1:9050",
}
resp = requests.get("https://"+url, proxies=proxies)
Monitor the outgoing DNS requests using Wireshark. If they are sent via the outbound network interface to your pre-configured DNS server (as opposed to the local interface, destination IP 127.0.0.1, port 9050), a DNS leak is occurring.
System Information
$ python -m requests.help
{
"chardet": {
"version": null
},
"charset_normalizer": {
"version": "2.0.12"
},
"cryptography": {
"version": ""
},
"idna": {
"version": "3.3"
},
"implementation": {
"name": "CPython",
"version": "3.10.2"
},
"platform": {
"release": "21.4.0",
"system": "Darwin"
},
"pyOpenSSL": {
"openssl_version": "",
"version": null
},
"requests": {
"version": "2.27.1"
},
"system_ssl": {
"version": "101010df"
},
"urllib3": {
"version": "1.26.8"
},
"using_charset_normalizer": true,
"using_pyopenssl": false
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with the provided requests.get example, an Apple Silicon pyenv Python, and the Tor SOCKS5 proxy at 127.0.0.1:9050; monitor DNS traffic with Wireshark. Trace urllib.request.proxy_bypass and the Requests/PySocks proxy path, and consider the issue done when socks5h DNS queries go through the SOCKS proxy rather than the client’s configured DNS server.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100