apache / apache/airflow

Add a way to differentiate between a task/DAG-level callback by context

Open
#61,119 4 comments 0 reactions 0 assignees View on GitHub
area:core kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 10h
Merged PRs (30d)
483

Description

### Description

Currently, the `context` passed to a callback defined on `DAG` level looks pretty much the same as a `context` passed to a task callback. Thus, we the callback's code doesn't have a way to differentiate between the two cases (from experimentation I noted that the task-level callback has an `exception` field while the DAG callback doesn't, but it doesn't look reliable enough).

I expected the DAG-level callback to not have a `task_instance` field, but apparently it points to the last task executed.

I propose injecting a new field into the context that points to the parent object (TI or DAG) that caused the callback to be called / enum that signifies whether it's a task or a dag level callback.

Current workaround is writing a callback class that has a boolean flag and use `partial` when initializing it for DAG/task.

### Use case/motivation

In my callback code I want to know which kind of callback was triggered and handle a failure accordingly.

### Related issues

https://github.com/apache/airflow/issues/26760

### Are you willing to submit a PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing how context is built and passed to DAG-level and task-level callbacks, comparing the documented fields with the reported task_instance behavior. Done means callback code can reliably identify which scope triggered it, with the chosen context field or enum behaving consistently for both callback levels.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.