apache / apache/airflow

Add ability to set context for airflow triggers

Open
#51,887 4 comments 0 reactions 0 assignees View on GitHub
area:Triggerer kind:feature needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

Add ability to set context for airflow triggers that can be used by all the python modules used in the triggerer code.

### Use case/motivation

In our airflow setup, we run the same dag for multiple teams. The context for each team like team ID is set at runtime using environment variables. The team id is used to access team specific services like db, blobs etc.

The reason for using environment variable is the convenience of accessing the value in all python modules without having to pass it as a variable to all methods.

This is working fine for running normal tasks in celery workers.

When it comes to triggerer, we are not able to achieve this using environment variables (or threading.local()), as triggerer is a single process, single thread service. Tried using contextvars, but the values set is not accessible from all modules.

Is there any way to achieve this?

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

The issue names the triggerer code and Python modules but no specific files, tests, or entry points. Start by locating the triggerer implementation and reviewing how context is currently accessed; done means a defined way to set per-team trigger context that is available to the relevant modules.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.