Drop airflow-core's own dev-group dependency on 6 providers
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
## What
`airflow-core/pyproject.toml`'s own `dev` dependency-group hardcodes 6 providers so that `uv sync --project airflow-core` (a scoped, core-only sync) can still run the full `airflow-core` test suite:
```toml
[dependency-groups]
dev = [
"apache-airflow-core[all]",
"apache-airflow-ctl",
"apache-airflow-devel-common",
"apache-airflow-task-sdk",
# TODO(potiuk): eventually we do not want any providers nor apache-airflow extras to be needed for
# airflow-core tests
"apache-airflow[pandas,polars]",
"apache-airflow-providers-amazon",
"apache-airflow-providers-celery",
"apache-airflow-providers-cncf-kubernetes",
"apache-airflow-providers-fab>=2.2.0",
"apache-airflow-providers-git",
"apache-airflow-providers-ftp",
]
```
There's an inline TODO already acknowledging this should go away "eventually," but no tracking issue exists for it.
## Why this matters
This is the remaining piece of the "standalone core testing" goal from #60770. That issue reported that `uv sync --project airflow-core` succeeds but the test suite doesn't actually run clean in that scoped environment. #71637 fixed the concrete, reproducible failures (a missing shared-package import and 11 tests that hard-coded full-provider-set assumptions), so the test suite now passes in this scoped sync — but only because these 6 providers are still installed alongside airflow-core's own dependencies. The environment isn't actually provider-free yet.
## Scope of the follow-up
1. Identify which `airflow-core` tests genuinely need `amazon`, `celery`, `cncf-kubernetes`, `fab`, `git`, and `ftp` (e.g. via `apache.hive`-style plugin tests, FAB-based auth-manager tests, connection form widgets that only render via `flask_appbuilder`, etc.).
2. For each: either move the test into the relevant provider's own test suite (if it's really testing provider behavior that leaked into `airflow-core`), or make it skip/adapt gracefully when that specific provider isn't installed (matching the pattern used for the tests fixed in #71637).
3. Once nothing in `airflow-core/tests` needs a provider unconditionally, drop the 6 provider entries (and `apache-airflow[pandas,polars]`) from `airflow-core/pyproject.toml`'s `dev` group and remove the TODO comment.
4. Update `contributing-docs/07_local_virtualenv.rst`'s existing note about standalone core development not being fully supported, once resolved.
This is a larger, multi-PR migration (touching many test files across `fab`/`amazon`/`celery`/`cncf.kubernetes`/`git`/`ftp`-adjacent behavior) rather than a single mechanical change, which is why it's split out from #60770 instead of being done in the same PR.
related: #60770
Contributor guide
Research direction
Start with airflow-core/pyproject.toml, airflow-core/tests, and the standalone-core testing changes from #71637. Identify tests that unconditionally need the six listed providers or apache-airflow extras, then move or adapt them as appropriate and verify the scoped core-only test suite passes without them. Finish by removing those dependencies and the TODO, and update contributing-docs/07_local_virtualenv.rst.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, developer-experience, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100