pydantic / pydantic/httpx2

Respect system proxy exclusions.

Open
#587 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.5k
Forks
76
Avg merge
8h 59m
Merged PRs (30d)
24

Description

Originally opened by @tomchristie on 2021-03-25 13:22:50 in encode/httpx

  • Initially raised as discussion #1513

We're currently leaning on urllib.request.getproxies() to determine the system proxy setup, and setup which mounts should be a proxy transport and which should be a regular transport.

However, we're not using urllib.request.proxy_bypass(host).

This all works as expected when environment settings are being used. HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY. In that case we're reading NO_PROXY, and ensuring anything hostname patterns there are mounted as a regular transport...

https://github.com/encode/httpx/blob/68cf1ff88a7f6c0b084bb62455c043aa503713ef/httpx/_utils.py#L304-L320

However, in the case when none of those environment variables are set getproxies() instead falls back to system proxy configuration. For windows this is registry based. ProxyEnable and ProxyOverride. For Mac this is sysconf based.
In those cases, we're correctly getting the configured proxies, but we aren't dealing with proxy exclusions.

We'd like to be able to setup these exclusions with our neat hostname pattern matched mounts system, which actually
means we can't just fallback to urllib.request.proxy_bypass(host), because that needs to be called per-host.

So, first steps...

  • What exactly is the format of the windows registry ProxyOverride field?
  • What exactly is the format of the "exceptions" field returned by from _scproxy import _get_proxy_settings()?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with httpx/_utils.py lines 304-320 and the existing getproxies()/NO_PROXY handling. Investigate the Windows ProxyOverride format and the exceptions field from _scproxy._get_proxy_settings(), then determine how those exclusions can fit the hostname-pattern mount system. Done means system proxy exclusions work when proxy settings come from Windows or macOS configuration, not only environment variables.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.