apache / apache/airflow

Custom deadline references registered as TYPES.DAGRUN_QUEUED are not re-anchored when a DagRun is cleared

Open
#71,747 1 comment 0 reactions 0 assignees View on GitHub
area:core area:deadline-alerts kind:bug
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core

### Apache Airflow version

3.3.0+

### What happened and how to reproduce it?

The evaluation-timing category passed to `@deadline_reference` is never persisted, and the one place that needs it doesn't actually use it, relying on identifying queued-anchored deadlines by hardcoded class name instead. Custom references are therefore silently excluded.

Clearing a DagRun re-queues it and refreshes `queued_at` (`DagRun.set_state`), and `_recalculate_dagrun_queued_at_deadlines` in `models/taskinstance.py` re-anchors the affected deadlines. Its selection criterion is a string comparison (line 244):

```python
DeadlineAlertModel.reference[ReferenceModels.REFERENCE_TYPE_FIELD].as_string()
== ReferenceModels.DagRunQueuedAtDeadline.__name__,
```

But `reference_type` holds the reference's own bare `__name__`, because
`BaseDeadlineReference.reference_name` returns `self.__class__.__name__` and the base `serialize_reference` emits `{REFERENCE_TYPE_FIELD: self.reference_name}`. Confirmed against stored rows:

```
reference_type | class_path
--------------------------+---------------------------------------
DagRunQueuedAtDeadline | <- built-in, matches
DagRunLogicalDateDeadline |
CloseOfBusinessDeadline | cob_reference.CloseOfBusinessDeadline <- custom, never matches
```

So a custom reference decorated `@deadline_reference(DeadlineReference.TYPES.DAGRUN_QUEUED)` can never satisfy that filter, no matter what it was registered as. `register_custom_reference` appends the class to the in-memory `TYPES.DAGRUN_QUEUED` tuple in whichever process ran the Dag file, and nothing writes that category to `DeadlineAlertModel`, so **the row the query reads has no timing field to filter on even in principle.**

To reproduce:

1. Register a custom reference with `@deadline_reference(DeadlineReference.TYPES.DAGRUN_QUEUED)` whose `_evaluate_with` returns `queued_at` plus something.
2. Put it on one Dag and `DeadlineReference.DAGRUN_QUEUED_AT` on another.
3. Trigger both, record `deadline.deadline_time`.
4. Clear both runs, and compare. The built-in is re-anchored to the new `queued_at`; the custom one is not.

### What you think should happen instead?

The registered timing category should determine which deadlines are re-anchored, so that a custom queued-anchored reference behaves like the built-in one.

1. The `DeadlineReference.TYPES` value needs to be added to the serialized reference, and
2. the check in `_recalculate_dagrun_queued_at_deadlines` which currently says

```python
DeadlineAlertModel.reference[ReferenceModels.REFERENCE_TYPE_FIELD].as_string()
== ReferenceModels.DagRunQueuedAtDeadline.__name__,
```

needs to check that value instead of the `__name__`.

### Operating System

_No response_

### Deployment

None

### Apache Airflow Provider(s)

_No response_

### Versions of Apache Airflow Providers

_No response_

### Official Helm Chart version

Not Applicable

### Kubernetes Version

_No response_

### Helm Chart configuration

_No response_

### Docker Image customizations

_No response_

### Anything else?

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start with _recalculate_dagrun_queued_at_deadlines in models/taskinstance.py, then trace BaseDeadlineReference.serialize_reference, reference_name, register_custom_reference, and the DeadlineAlertModel reference fields. Reproduce the built-in and custom queued-anchored deadlines, clear both DagRuns, and compare their deadline times. Done means the serialized timing category identifies custom queued references and clearing re-anchors them to the refreshed queued_at.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.