apache / apache/airflow

TimeDeltaSensor suggestion

Open
#33,373 5 comments 0 reactions 0 assignees View on GitHub
kind:feature provider:standard
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Description

Currently, in TimeDeltaSensor its `delta` interval starts from [its dagrun creation](https://github.com/apache/airflow/blob/main/airflow/sensors/time_delta.py#L44) but not from when the task's actually been running.

### Use case/motivation

Imagine we've got some tasks before the TimeDeltaSensor task.
In order to use TimeDeltaSensor we need to know their expected execution time and add this time to the sensor's delta parameter. Ok, it's not good for using but can be acceptable. But what if their expected execution time may vary for some reason from time to time? How to calculate the delta to be passed in sensor?

So, is there any reason it's done this way? Wouldn't it be better if `delta` was attached to its task instance, which seems to be more intuitive. smth like this:
```
L44 target_dttm = context['task_instance'].start_date
```

### Related issues

_No response_

### Are you willing to submit a PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by reading airflow/sensors/time_delta.py around line 44 and compare the current DAG-run creation reference with the task instance start date described in the issue. The work is done when TimeDeltaSensor's delta is measured from the task's actual start rather than the DAG run creation time, with the resulting behavior verified in the relevant sensor tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.