apache / apache/airflow

Indicate warnings or other "non-failing" indicators in Airflow UI and callbacks

Open
#42,757 2 comments 3 reactions 0 assignees View on GitHub
area:UI kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Sometimes a task shouldn't fail but could contain a warning that the user should know about and fix; for example, a deprecation warning or a user-logged warning. These warnings don't "bubble up" anywhere in the Airflow UI.

Making warnings more visible could have the impact of making upgrades (like the upcoming 3.0 upgrade) much easier for developers.

Possible but not optimal current solutions:

- You can log a warning-level task log, but aggregating all warning-level task logs in the Airflow UI is (as far as I know) not possible.
- You could also customize the log handler to handle WARNING-level messages across all DAGs, but sending these messages to Slack or otherwise might be expensive within this process
- Aggregate using third party tool

A solution could look like all otherwise successful task instances in a DAG run labeled by a yellow (or other color) dot instead of a green dot. It could also help to aggregate warning-level or error-level messages in the UI.

Implementation wise - Maybe there could be a separate task instance state (warning) or different UI indicator on the current success state that could be leveraged to visually indicate this situation. If it were a separate warning state, an on_warning_callback could also be added to send more information to the user's existing Slack alert workflow.

### Use case/motivation

Better indication that something in a task went wrong, or something happened the user should be notified about

### Related issues

@RNHTTR recommend I copy here from this discussion:

https://github.com/apache/airflow/discussions/42188

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reading the issue and the linked Airflow discussion, then map how task warnings, task-instance states, the UI, and callbacks currently relate. Done would require an agreed design for representing non-failing warnings and a clearly scoped implementation plan, since the issue does not name files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, full-stack
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.