apache / apache/airflow

Multi-team: pass the bundle's team_name to secrets backends during DAG parsing

Open
#70,495 2 comments 0 reactions 0 assignees View on GitHub
area:core area:dag-bundles area:secrets kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Description

Make secrets backends receive the DAG bundle's team_name during DAG file parsing, the same way they receive it during task execution.

This makes team-scoped resolution consistent between parse time and run time, so a team-scoped backend (for example one routing to a per-team Vault by team_name) can resolve config for a bundle's DAGs while they are being parsed.

Today team_name is always None at parse time.

### Use case/motivation

Multi-team dynamic DAG generation that reads a team-scoped connection or variable at parse time. A bundle belongs to a team, but a team-scoped secrets backend can resolve config for that bundle's tasks and not for parsing of the same bundle's files, because team_name is None there.

The bundle to team mapping needed to close the gap is already in the DAG processor.

### Related issues

https://github.com/apache/airflow/issues/65530

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start in the DAG processor and trace how the bundle-to-team mapping and secrets backend context are passed during DAG file parsing. Compare that path with task execution, then locate or add coverage for a team-scoped backend resolving a connection or variable during parsing. Done means parsing receives the bundle's team_name instead of None while execution behavior remains consistent.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.