ClickHouse / ClickHouse/clickhouse-rs

Recognize and support proxies

Open
#416 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
559
Forks
172
Avg merge
1d 19h
Merged PRs (30d)
3

Description

### Use case
The `reqwest` crate supports system HTTP(S) forward proxies configured by environment variable: https://docs.rs/reqwest/latest/reqwest/#proxies

* `HTTP_PROXY`/`http_proxy`
* `HTTPS_PROXY`/`https_proxy`
* `ALL_PROXY`/`all_proxy`

However, since we use the `hyper` crate directly, we don't get support for proxies out of the box and we have not implemented it manually yet.

### Describe the solution you'd like

Switching to the `reqwest` crate would automatically give us proxies as well as proper HTTP connection pooling (and would transitively close #313).

I had previously recommended this but there was the concern that we would lose performance because we don't have as tight of control over the HTTP connection. This assumption should really be tested, however.

### Describe the alternatives you've considered

The equivalent proxy support is available in the `hyper-util` crate: https://docs.rs/hyper-util/latest/hyper_util/client/proxy/matcher/struct.Matcher.html#method.from_env

This _should_ give us the exact same behavior as `reqwest` (it's [what `reqwest` uses under the hood](https://github.com/seanmonstar/reqwest/blob/5d5bf355744b181d31533501133ad9fbf99e8849/src/proxy.rs#L7)) but we'd still have to implement and test this ourselves to be certain it works correctly. Using `reqwest` would push that technical debt upstream.

### Additional context

Requested by @keltia on the ClickHouse Community Slack: https://clickhousedb.slack.com/archives/CU478UEQZ/p1775582175943629

Contributor guide

Open the contributing guide

Research direction

Start by examining the current direct use of the hyper crate, then compare the reqwest proxy behavior with hyper-util's client/proxy/matcher::Matcher::from_env entry point. The work is done when HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY settings are supported with tests covering the expected behavior and the connection-performance concern is evaluated.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.