apache / apache/airflow

Use only single execution date in ExternalTaskSensor

Open
#12,577 2 comments 0 reactions 0 assignees View on GitHub
kind:feature provider:standard
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

**Description**

In general we encourage operators to be designed to perform an atomic action. However, `ExternalTaskSensor` seems to be not alway atomic. Users are able to pass `execution_date_fn` which can return multiple dates. If that happens the operator will check state not of single DAG/task but of multiple instances. Even if that happens users will get 1/0 response - task/dag failed or not.

In my opinion the proper way to handle the multiple dates case would be to have multiple instances of the sensor, each checking for a task/dag for a given date.

**Use case / motivation**

Improve atomicity of `ExternalTaskSensor`.

**Related Issues**

_Originally posted by @turbaszek in https://github.com/apache/airflow/pull/12574#discussion_r528967901_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.