apache / apache/airflow

Log exceptions as warnings till the final attempt

Open
#32,246 10 comments 1 reaction 0 assignees View on GitHub
good first issue kind:feature
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

We have an integration with sentry on Google cloud composer.
We would like to be alerted for any exceptions happening in airflow , but don't want to be alerted when those exceptions would be retried. We tried setting alert conditions on number of errors, but it does not solve the problem as we have hundreds of instances of a DAG running.

Requirements:
1. Log as warning when try_number is less than equal to max_tries.
2. Log as an error in the final try.

### Use case/motivation

_No response_

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by locating the Airflow exception-logging path that has access to try_number and max_tries. Verify the current behavior for retried and final attempts, then confirm that retryable exceptions are warnings and the final attempt is an error, with tests covering both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, observability
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.