apache / apache/airflow

Handling of the DagRun callback log should be visible on the UI somewhere

Open
#32,379 9 comments 5 reactions 0 assignees View on GitHub
kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

When a DagRun handles its callbacks, the logs are only available in the DAGFileProcessorProcess log location. I want to be able to view these logs in the Airflow UI or at least a subset of the logs generated by the callbacks.

In addition, I would like a surefire way to confirm on the Airflow UI that the scheduler sent the DagCallbackRequest and the DAGFileProcessorProcess handled the callback requqest.

### Use case/motivation

I had written a DAG with `on_success_callback` set and I noticed that certain dagruns did not generate the corresponding `on_success_callback` in the DAGFileProcessorProcess logs.

### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by tracing how the scheduler sends the DagCallbackRequest and how the DAGFileProcessorProcess handles it, then identify where the Airflow UI exposes DagRun logs or callback status. Done should include a visible way to inspect callback logs or a defined subset and a reliable UI indication that the request was sent and handled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, frontend, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.