DAG Bundle/User Impersonation Causes Permission Issues in Resource Preparation
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
3.1.5
### If "Other Airflow 3 version" selected, which one?
_No response_
### What happened?
When running Git-based DAG bundles in Airflow with user impersonation (`run_as_user`), tasks may fail due to permission errors on resources like lock files and tracking directories. This occurs because bundles are initialized by the main airflow user before impersonation, making it necessary for the impersonated user to access these resources. Attempts to mitigate this in PRs [#60270](https://github.com/apache/airflow/pull/60270), [#60278](https://github.com/apache/airflow/pull/60278), and [#60280](https://github.com/apache/airflow/pull/60280) involve configuration changes and runtime warnings, but the underlying architectural problem remains unaddressed.
### What you think should happen instead?
Resource preparation for bundles should be performed post-impersonation—or in a context where only the appropriate user has write access. Impersonated users should not need elevated permissions or write access to resources initialized by the airflow service user. The fix should be holistic, ideally centralizing resource management with privilege separation and avoiding relaxed file permissions or risky group sharing.
### How to reproduce
1. Set up an Airflow deployment using Git-based DAG bundles and enable user impersonation with the `run_as_user` config.
2. Configure a DAG that fetches and interacts with resources stored in a shared repository (e.g., involving lock files, tracking directories).
3. Trigger the DAG with a task that switches user context.
4. Observe that the task fails with permission errors accessing bundle-related files or directories.
5. Attempt to mitigate by setting group-writable permissions or updating git's `safe.directory`; note that this works but relaxes security.
### Operating System
Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else?
- Short-term solutions as seen in recent PRs involve group-writable directories, explicit warnings, and setting `safe.directory`. However, these may not be viable for all production environments, such as containerized or multi-tenant stacks.
- Recommending a long-term refactor, possibly using a supervisor or Execution API to manage resource preparation, with only read-access needed for impersonated task users.
- Would appreciate feedback, especially for edge-case deployments.
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Review PRs #60270, #60278, and #60280 alongside the run_as_user and Git-based bundle reproduction steps. Determine where bundle resource preparation occurs relative to impersonation, then define a privilege-separated approach whose completion is demonstrated by the reproduction succeeding without relaxed permissions or elevated access for the impersonated user.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- backend, data-engineering, security
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100