apache / apache/airflow

Proposal: Runtime CRUD API and UI support for DAG Bundles

Open
#61,645 7 comments 0 reactions 0 assignees View on GitHub
area:API area:dag-bundles area:UI kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

Airflow 3 introduced DAG Bundles configured via [dag_processor].dag_bundle_config_list.
Currently bundle definitions are static and only loaded from configuration at process startup.

This makes it difficult to manage bundles dynamically in environments where DAG sources are discovered or change over time.

It would be helpful to support:

- runtime CRUD of bundle definitions via the REST API (e.g. /api/v1/dag_bundles)
- persistence in the metadata DB
- automatic reload by DAG Processor(s)
- visibility and management from the Airflow UI (list, inspect, enable/disable, create/delete)

This would make bundles first-class operational resources instead of static configuration.

### Use case/motivation

I run Airflow with a GitHub OAuth/GitHub App that has access to many organizations and repositories (private + public).

Each git repository may contain DAGs under a workflows/ directory.

Repositories are added/removed over time and should be discovered automatically.

Expected behavior:

- dynamically detect accessible repositories
- create one bundle per repository
- refresh them automatically
- avoid hard-coded configuration
- avoid restarting Airflow when repositories change

Security and operational goals:

- per-repository isolation (one repo = one bundle)
- independent bundle versioning
- no long-lived per-repo credentials
- zero-downtime onboarding of new repositories

Today this requires regenerating config and restarting components, or aggregating multiple repositories into a single bundle, which reduces isolation, weakens security boundaries between repositories, and increases operational overhead.

A dynamic bundle registry and API would address this cleanly.

### Related issues

- https://github.com/apache/airflow/discussions/59799
- https://github.com/apache/airflow/issues/60630

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.