Allow taskflow tasks to simultaneously return an XCOM and mark themselves as failed.
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Description
Taskflow is designed to abstract away a lot of the base Airflow concepts like xcom, hence the task return values are converted to xcom automatically. Similarly you don't have to explicitly mark a task as failed, you can just raise an exception.
However there is one gap in this abstraction, namely when you want to do both at once. There is nothing in airflow that prevents a task from pushing an xcom and also failing, but taskflow provides no abstraction for this. At present you would have to explicitly use `push_xcom` and then `raise`. Therefore I suggest a new interface for this, which is a new class called something like `AirflowResultException`, which takes a value that will get pushed to xcom, but the task will still be failed. For example:
```python
class AirflowResultException(AirflowException):
def __init__(self, value: Any):
...
@task
def my_task(x: bool):
if x:
return "foo" # "foo" is pushed as an xcom, and the task status is set to succeeded
else:
raise AirflowResultException("bar") # "bar" is pushed as an xcom, and the task status is set to failed
```
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by tracing how the @task interface converts return values to XCom and how AirflowException or an explicit push_xcom followed by raise marks a task failed. Done means a task can raise the proposed result exception, preserve its value as XCom, and still finish in the failed state, with coverage for both return and exception paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100