apache / apache/airflow

Support deferring from taskflow tasks

Open
#33,663 3 comments 4 reactions 0 assignees View on GitHub
kind:feature kind:meta
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### Body

could be cool

you just have to make it so that the function gets re-entered

then you could have something like

```python
@task
def my_deferrable(event=None):
if event:
do something
else:
do something and then defer
```

There have been other ideas, doing something with yield

cc @uranusjr since we chatted about it at one point

### Committer

- [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Contributor guide

Open the contributing guide

Research direction

Start with the TaskFlow @task entry point and review how deferrable tasks are currently represented. Define and document the expected re-entry behavior, including how an event reaches the task after deferral, then verify the proposed approach against the example in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.