apache / apache/airflow

HttpAsyncHook replays connection extra headers across a cross-host redirect

Open
#70,164 0 comments 0 reactions 0 assignees View on GitHub
area:providers kind:bug provider:http security
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Apache Airflow Provider(s)

http

### What happened

`HttpAsyncHook.config` copies every non-reserved key of an HTTP Connection's `extra` field into the `aiohttp.ClientSession` headers, and the provider docs endorse that field as a place to carry credentials ("Login and Password authentication can be used along with any authentication method using headers. Headers can be given in json format in the Extras field").

`aiohttp` follows redirects by default and strips only the literal `Authorization` header when the redirect changes host. A secret carried under any other header name (`X-API-Key`, `Private-Token`, `apikey`, ...) is a plain session header and is replayed verbatim to every host in the redirect chain. Verified against aiohttp 3.14.1: `Authorization` is dropped on a cross-host redirect, `X-API-Key` is forwarded.

The destination does not have to be attacker-controlled for this to leak - an API that 302s downloads to a CDN or object store receives the connection's key.

### What you think should happen instead

Connection-supplied headers should get the same treatment `aiohttp` already gives `Authorization`: dropped once the redirect leaves the original host, preserved on same-host redirects.

### Follow-up work

https://github.com/apache/airflow/pull/70000 fixes the synchronous `HttpHook` by subclassing `requests.Session` and extending `rebuild_auth`, which is the hook `requests` exposes for exactly this. `aiohttp` has no equivalent per-redirect callback, so the async side needs a different approach - most likely disabling automatic redirect following in `HttpAsyncHook.run` and walking the chain manually so headers can be recomputed per hop, or moving the connection headers off the session and onto each request.

### Acceptance criteria

- A cross-host redirect from `HttpAsyncHook` does not forward Connection `extra` headers.
- A same-host redirect still forwards them.
- Regression test covering both directions, mirroring `test_connection_header_is_only_forwarded_on_a_same_host_redirect` in `providers/http/tests/unit/http/hooks/test_http.py`.

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with HttpAsyncHook.run and the existing same-host redirect regression test in providers/http/tests/unit/http/hooks/test_http.py. Trace how Connection extra headers enter the aiohttp session and how redirects are followed. Done means connection headers remain on same-host redirects but are not forwarded across hosts, with tests covering both directions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.