psf / psf/requests

urllib.request.proxy_bypass makes DNS request without using configured proxies when system has NO_PROXY configured at some level

Open
#6,086 15 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

When using Requests via a SOCKS5 proxy on one of the latest MacBook Pros (Apple Silicon), the DNS requests are not correctly passed to the proxy server even when using the "socks5h" scheme. Instead, the DNS resolution happens at the level of the client, causing a DNS leak.

This happens with various versions/combinations of Requests/PySocks/Python, the latter being the Apple Silicon versions obtained through pyenv. I have tested this using the local SOCKS5 proxy server provided by Tor.

In more detail, I tested the bug on various pyenv Python distributions, labeled 3.8 to 3.10 plus miniforge3, with Requests from v2.16.0 to v2.27.1, and PySocks from v1.5.7 to v1.7.1 (not every single version in between). In what follows, I'll use an example System Information output.

Also, I used Wireshark to monitor the DNS requests, and I used curl with the --proxy socks5h://127.0.0.1:9050 flag as a control, to make sure that the local Tor proxy was working properly and that the DNS leak is indeed specific to Python/Requests/PySocks.

I am not able to confirm whether the DNS leak also happens on Python versions obtained other than through pyenv. On an Intel MacBook using an Anaconda Python version and the latest Requests/PySocks, the leak does not occur.

Expected Result

When using the "socks5h" scheme, the DNS requests should be forwarded to the SOCKS proxy.

Actual Result

The DNS requests are sent from the client instead, causing a DNS leak.

Reproduction Steps

Install any Apple Silicon Python version from pyenv (see above for the versions I tested) and start Tor.

import requests

proxies = {
    "http": "socks5h://127.0.0.1:9050",
    "https": "socks5h://127.0.0.1:9050",
}

resp = requests.get("https://"+url, proxies=proxies)

Monitor the outgoing DNS requests using Wireshark. If they are sent via the outbound network interface to your pre-configured DNS server (as opposed to the local interface, destination IP 127.0.0.1, port 9050), a DNS leak is occurring.

System Information

$ python -m requests.help
{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.10.2"
  },
  "platform": {
    "release": "21.4.0",
    "system": "Darwin"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010df"
  },
  "urllib3": {
    "version": "1.26.8"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the provided requests.get example, an Apple Silicon pyenv Python, and the Tor SOCKS5 proxy at 127.0.0.1:9050; monitor DNS traffic with Wireshark. Trace urllib.request.proxy_bypass and the Requests/PySocks proxy path, and consider the issue done when socks5h DNS queries go through the SOCKS proxy rather than the client’s configured DNS server.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.