apache / apache/airflow

Consider swapping `httpx` usage for `httpx2`

Open
#70,522 3 comments 1 reaction 0 assignees View on GitHub
area:dependencies kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.x

### What happened and how to reproduce it?

At the time of writing, it appears that the `httpx` library might not be actively maintained. There are a number of bug-fixes for `httpx` which have not landed, but which are present in [`httpx2`](https://httpx2.pydantic.dev/) (forked from `httpx==0.28.1`).

Additionally, the maintainer of `starlette` (one of the key sub-dependencies of `FastAPI`) [has confirmed](https://github.com/pydantic/httpx2/discussions/980#discussioncomment-17030187) that starlette will migrate to `httpx2`. So it seems the migration from `httpx` in `FastAPI` is an eventuality.

### What you think should happen instead?

Any direct usage of `httpx` within airflow-core / the task-sdk / airflow's providers should probably migrate to httpx2.

The scope of this change is obviously large; but the migration of direct code reliance on `httpx` is [relatively straightforward](https://httpx2.pydantic.dev/migration/).

I'd imagine it would be good to add individual, well-defined / scoped sub-issues for this.

### Operating System

_No response_

### Deployment

_No response_

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

_No response_

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by inventorying direct httpx usage across airflow-core, the task-sdk, and Airflow providers, then read the httpx2 migration guide linked in the issue. The scope is broad, so identify well-defined sub-issues for individual areas. Done means the selected direct usages have migrated to httpx2 and their affected tests or checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.