Support updating DagRun note from notifier/callback context in Airflow 3
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Description
In Airflow 2, a common notifier/callback pattern was:
1. Post an alert to an external system such as Slack.
2. Persist a backlink or correlation identifier on the `DagRun` note.
3. Reuse that stored link later to update the original alert if the DagRun is cleared and eventually succeeds.
That migration path is not straightforward in Airflow 3.
In Airflow 3 callback/notifier code, `context["dag_run"]` is no longer the old SQLAlchemy `DagRun` contract that many Airflow 2 implementations relied on, and the Task SDK client does not appear to expose a supported helper for updating `DagRun.note` from callback/notifier runtime code.
There is already public API support for DagRun note patching, so this request is specifically about having a supported callback-safe/runtime-safe way to do that from Airflow 3 notifier code.
### Use case/motivation
A concrete use case is a DAG-level failure notifier that:
1. Fires once when the DagRun fails.
2. Collects failed task IDs for that run.
3. Posts a Slack message.
4. Stores the Slack thread URL in the DagRun note for traceability.
5. On a later callback (for example after the DagRun is cleared and succeeds), reads that stored link and updates the original Slack message instead of sending a brand new one.
The "list failed tasks" part is portable in Airflow 3 via `context["task_instance"].get_task_states(...)` in a real callback execution path.
The missing piece is the note update. In Airflow 2 this was commonly done with ORM/session access such as `DagRunNote`, `dag_run.dag_run_note`, and `provide_session`. In Airflow 3 that pattern is not portable, but I could not find a supported replacement in notifier/callback context.
### Related issues
- #51816 is related only in the sense that local `dag.test()` / `airflow dags test` debugging can be misleading for callback runtime APIs. It does not solve the DagRun note update gap described here.
### Requested outcome
One of the following would unblock Airflow 2 -> 3 notifier migrations cleanly:
1. Add a supported Task SDK / callback-context helper to update `DagRun.note` from notifier code.
2. Expose a documented runtime client operation for patching the current DagRun note from callback code.
3. If this is intentionally unsupported, document the recommended replacement pattern for storing callback correlation metadata on a DagRun in Airflow 3.
### Environment where this came up
- Apache Airflow 3.1.8
- DAG-level `on_failure_callback`
- Local Astro / Astronomer runtime during migration testing
### Minimal callback shape
```python
from airflow.notifications.basenotifier import BaseNotifier
class ExampleNotifier(BaseNotifier):
def notify(self, context):
dag_run = context["dag_run"]
ti = context["task_instance"]
task_states = ti.get_task_states(
dag_id=dag_run.dag_id,
run_ids=[dag_run.run_id],
)
failed_tasks = [
task_id
for task_id, state in task_states.get(dag_run.run_id, {}).items()
if state in {"failed", "upstream_failed"}
]
# Supported in Airflow 3: derive failed task list.
print(failed_tasks)
# Missing piece: supported way to persist a backlink such as a Slack URL
# onto the DagRun note from this notifier/callback context.
```
### Why I think this belongs in OSS Airflow
This appears to be a migration and API-surface gap rather than a managed-service issue:
- Airflow 3 callback context behavior changed relative to many Airflow 2 implementations.
- Public DagRun note patching exists, so the underlying capability is present.
- The missing piece is a supported, documented path for callback/notifier code to update the current DagRun note.
Thanks for taking a look.
Contributor guide
Assessment
This issue has not been assessed yet.