Add ability to set context for airflow triggers
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
Add ability to set context for airflow triggers that can be used by all the python modules used in the triggerer code.
### Use case/motivation
In our airflow setup, we run the same dag for multiple teams. The context for each team like team ID is set at runtime using environment variables. The team id is used to access team specific services like db, blobs etc.
The reason for using environment variable is the convenience of accessing the value in all python modules without having to pass it as a variable to all methods.
This is working fine for running normal tasks in celery workers.
When it comes to triggerer, we are not able to achieve this using environment variables (or threading.local()), as triggerer is a single process, single thread service. Tried using contextvars, but the values set is not accessible from all modules.
Is there any way to achieve this?
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
The issue names the triggerer code and Python modules but no specific files, tests, or entry points. Start by locating the triggerer implementation and reviewing how context is currently accessed; done means a defined way to set per-team trigger context that is available to the relevant modules.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100