TaskFlow unexpectedly unwinds 'dict'
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
main (development)
### What happened
if using the TaskFlow API and one does the following
```
@task()
def MyFunc(myarg: int) -> dict:
return dict({x: "y", z: "x"})
```
Airflow automatically unwinds `dict` automatically into separate variables `x` and `z` and if you do not want that kind of bahavior you will need to set `multiple_outputs=False`. While this is in the documentation, it is rather non intuitive requiring someone to be aware of the documentation that the `default` of `multiple_outputs` changes depending the return type of your function. Moreover, it is unexpected behavior for people coming from plain python (e.g. someone converting their functions to Airflow tasks) and wanting to do:
```
@task()
def f1(myarg: int) -> dict:
return dict({x: myarg, z: "x"})
@task
def f2(in: dict):
print(in)
a = f1(10)
f2(a)
```
Which would suddenly have issues in unittesting.
cc @josh-fell @BasPH @uranusjr
### What you think should happen instead
Airflow should only unwind dicts if explicitly asked to do so.
### How to reproduce
_No response_
### Operating System
Not relevant
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the TaskFlow API examples using @task and a function annotated to return dict, then trace how multiple_outputs is inferred from the return annotation. Reproduce the f1/f2 example and verify that dictionaries remain single task outputs unless multiple_outputs is explicitly enabled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100