psf / psf/requests

Transport adapters not honored with proxy

Open
#6,205 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

hi,

making a poor socks server at localhost on port 8001 with ssh/sshd running at localhost :

@term1$ ssh -N -D localhost:8001 localhost

making a self-signed certificate for testing and debuging negociations on server side with s_server@openssl :

@term2$ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes 
        while :
        do
            echo -e "HTTP/1.0 200 OK\nContent-Length: 0\n\n" |
            openssl s_server -4 -cert cert.pem -key key.pem -accept 4433 -msg
        done

getting requests informations :

@term3$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.10"
  },
  "platform": {
    "release": "5.10.0-1057-oem",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010106f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.8"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}

running requests simply :

@term3$ python <<~~~
import requests
url = "https://localhost:4433/"
session = requests.Session()
response = session.head(url=url, verify=False)
~~~

TLS1.3 by default :

@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...

running requests with transport adapter only :

@term3$ python <<~~~
import ssl
import requests
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
class TestHTTPAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(
            num_pools=connections,
            maxsize=maxsize,
            block=block,
            ssl_version=ssl.PROTOCOL_TLSv1_2
        )
url = "https://localhost:4433/"
session = requests.Session()
session.mount(url, TestHTTPAdapter())
response = session.head(url=url, verify=False)
~~~

TLS1.2 is OK 🙂 :

@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

running requests with transport adapter and socks proxy :

@term3$ python <<~~~
import ssl
import requests
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
class TestHTTPAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(
            num_pools=connections,
            maxsize=maxsize,
            block=block,
            ssl_version=ssl.PROTOCOL_TLSv1_2
        )
url = "https://localhost:4433/"
session = requests.Session()
session.mount(url, TestHTTPAdapter())
proxies={"https":"socks5://localhost:8001"}
response = session.head(url=url, proxies=proxies, verify=False)
~~~

TLS1.2 is KO 🙁 :

@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...

requests has the same behavior whether pyopenssl is present/used or not.
HTTP/HTTPS proxies not tested : sorry.

regards, lacsaP.


note that httpie works as expected :

@term4$ https --verify no https://localhost:4433
HTTP/1.0 200 OK
Content-Length: 0
@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...
@term4$ https --ssl tls1.2 --verify no https://localhost:4433
HTTP/1.0 200 OK
Content-Length: 0
@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...
@term4$ https --ssl tls1.2 --verify no --proxy https:socks5://localhost:8001 https://localhost:4433
HTTP/1.0 200 OK
Content-Length: 0
@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

as well as urllib3 :

@term5$ python <<~~~
import urllib3
https = urllib3.PoolManager(cert_reqs='CERT_NONE')
r = https.request('GET', 'https://localhost:4433')
~~~
@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...
@term5$ python <<~~~
import ssl, urllib3
https = urllib3.PoolManager(cert_reqs='CERT_NONE', ssl_version=ssl.PROTOCOL_TLSv1_2)
r = https.request('GET', 'https://localhost:4433')
~~~
@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...
@term5$ python <<~~~
import ssl
from urllib3.contrib.socks import SOCKSProxyManager
socks = SOCKSProxyManager('socks5h://localhost:8001/', cert_reqs='CERT_NONE', ssl_version=ssl.PROTOCOL_TLSv1_2)
r = socks.request('GET', 'https://localhost:4433')
~~~
@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the custom HTTPAdapter, the SOCKS proxy, and the TLS 1.2 setting shown in the report. Compare the direct and proxy request paths, then verify that the configured TLS version is honored through the proxy while preserving the existing request behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.