apache / apache/airflow

airflow-ctl: migrate to httpx 1.x (after rewrite)

Open
#65,609 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

## Context

httpx 1.0 (currently \`1.0.dev3\` on PyPI) is a ground-up rewrite rather than a normal major bump. Inspecting \`httpx==1.0.dev3\`:

- The entire public exception hierarchy is gone — only \`ProtocolError\` remains. No more \`HTTPError\`, \`RequestError\`, \`TransportError\`, \`HTTPStatusError\`, \`TimeoutException\`, \`ConnectError\`, \`ReadError\`, \`WriteError\`, \`ProxyError\`, \`TooManyRedirects\`, \`DecodingError\`, \`StreamError\`, \`InvalidURL\`, etc.
- \`Response.raise_for_status()\` → needs a replacement check API.
- \`AsyncClient\` / \`Client\` / \`Transport\` / \`Auth\` surface is redesigned.
- Many new primitives (\`serve_http\`, \`run\`, \`DuplexStream\`, \`HTTPParser\`, \`ConnectionPool\`) — it ships as both a client **and** a server.

## Current airflow-ctl usage

\`airflow-ctl/src/airflowctl/api/client.py\` and \`airflow-ctl/src/airflowctl/api/operations.py\` use:

- \`httpx.Client\`, \`httpx.AsyncClient\`
- \`httpx.HTTPStatusError\` (including a subclass \`ServerResponseError(httpx.HTTPStatusError)\`)
- \`httpx.ConnectError\`, \`httpx.ReadError\` in retry/error-handling logic
- \`Response.raise_for_status()\`
- \`httpx.MockTransport\` in tests (\`airflow-ctl/tests/airflow_ctl/api/test_operations.py\`)

## Interim mitigation

[#65607](https://github.com/apache/airflow/pull/65607) caps \`httpx<1.0\` so users who pass \`--pre\` / \`--prerelease=allow\` do not pull the rewrite and get a crashing install. That unblocks 0.1.4rc3 but does not address the migration.

## What this issue tracks

Migrating airflow-ctl to httpx 1.x once it stabilizes:

- [ ] Audit \`httpx\` API usage in \`api/client.py\`, \`api/operations.py\`, and tests.
- [ ] Design replacements for \`ServerResponseError(httpx.HTTPStatusError)\` and the retry/error-handling paths (likely a custom exception hierarchy that is not tied to an httpx public one).
- [ ] Port \`httpx.MockTransport\`-based tests to whatever 1.x exposes for test-time HTTP mocking.
- [ ] Update \`httpx>=0.27.0,<1.0\` → \`>=1.0\` in \`airflow-ctl/pyproject.toml\` once the rewrite is released and adopted.
- [ ] Verify \`airflowctl\` remote-CLI workflows against the 1.x client surface (auth flows, streaming downloads for logs, timeout semantics).

## Acceptance

\`airflowctl\` installs cleanly with \`pip install --pre apache-airflow-ctl\` on httpx 1.x and all existing airflow-ctl tests pass.

---

Opened as part of the 0.1.4rc3 release prep — see #65497 for rc2 testing feedback that surfaced the incompatibility.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.