psf / psf/requests

Session does not consistently respects proxy environment variables.

Open
#5,677 9 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
54.3k
Forks
10.4k
Avg merge
16h 43m
Merged PRs (30d)
3

Description

There are inconsistencies when working with proxy environment variables using Session objects.

When using Session#request or verb methods (get, post, put), those work correctly and use the proxy when they are present.

However Session#send(PreparedRequest) does not respect the variables by default, but does when it handles redirect responses, e.g. 303.

I've written a test class to present this behavior.

Expected Result

All methods in Session to respect the proxy variable.

Actual Result

Session#send(PreparedRequest) does not respect proxy environment variables.

Reproduction Steps

Test case 1 fails with a successful request.

import requests
import pytest

# Copied from requests/tests/utils.py for making this a standalone
import os
import contextlib

@contextlib.contextmanager
def override_environ(**kwargs):
    save_env = dict(os.environ)
    for key, value in kwargs.items():
        if value is None:
            del os.environ[key]
        else:
            os.environ[key] = value
    try:
        yield
    finally:
        os.environ.clear()
        os.environ.update(save_env)

class TestSession:

    def test_respect_proxy_env_on_send(self):
        session = requests.Session()

        # Does not respect the proxy configuration and ignores the proxy
        request = requests.Request(
            method='GET', url='https://www.google.com/'
        )

        with override_environ(https_proxy='http://example.com'):
            try:
                session.send(request.prepare())
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)
        
    def test_respect_proxy_env_on_send_with_redirects(self):
        session = requests.Session()

        # Returns a 303 to www.google.com and respects the proxy
        request = requests.Request(
            method='GET', url='https://google.com/'
        )

        with override_environ(https_proxy='http://example.com'):
            try:
                session.send(request.prepare())
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)

    def test_respect_proxy_env_on_get(self):
        session = requests.Session()

        with override_environ(https_proxy='http://example.com'):
            try:
                # No redirects
                session.get('https://www.google.com')
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)

    def test_respect_proxy_env_on_request(self):
        session = requests.Session()

        with override_environ(https_proxy='http://example.com'):
            try:
                # No redirects
                session.request(method='GET', url='https://www.google.com')
                assert False, "The proxy is invalid this request should not be successful."
            except requests.exceptions.ProxyError as e:
                assert 'Cannot connect to proxy' in str(e)

System Information

$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.10"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.1"
  },
  "platform": {
    "release": "5.9.8-100.fc32.x86_64",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.25.0"
  },
  "system_ssl": {
    "version": "100020ff"
  },
  "urllib3": {
    "version": "1.26.2"
  },
  "using_pyopenssl": false
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing Session#send, Session#request, and the verb methods to compare how proxy environment variables are applied. Use the supplied TestSession cases, especially the direct PreparedRequest and redirect scenarios; done means all Session entry points consistently respect the proxy variable and the tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.