XCom dictionary key order not preserved
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
3.1.0
### If "Other Airflow 2/3 version" selected, which one?
_No response_
### What happened?
When passing a dictionary through XCom (at least as a return-value), key order is not preserved. The keys appear to be sorted alphabetically when inspected from a consuming task.
### What you think should happen instead?
Since Python 3.7, dictionaries preserve the insertion order of keys. JSON dumping and loading likewise preserve this also. Since Airflow is well past that version, I would expect it to simply inherit this behavior.
### How to reproduce
Minimal DAG:
```python
from airflow.decorators import dag, task
@dag(render_template_as_native_obj=True)
def d1():
@task
def create() -> dict:
return {
'C': 1,
'B': 2,
'A': 3,
}
@task
def render(d: dict):
print(d)
render(create())
d1()
```
The output:
```
INFO - {'A': 3, 'B': 2, 'C': 1} source=task.stdout
```
### Operating System
MacOS Tahoe 26.0.1 (25A362)
### Versions of Apache Airflow Providers
I doubt any of these are relevant, but just in case:
apache-airflow==3.1.0
apache-airflow-core==3.1.0
apache-airflow-providers-amazon==9.14.0
apache-airflow-providers-celery==3.12.3
apache-airflow-providers-cncf-kubernetes==10.8.1
apache-airflow-providers-common-compat==1.7.4
apache-airflow-providers-common-io==1.6.3
apache-airflow-providers-common-messaging==2.0.0
apache-airflow-providers-common-sql==1.28.1
apache-airflow-providers-docker==4.4.3
apache-airflow-providers-elasticsearch==6.3.3
apache-airflow-providers-fab==2.4.3
apache-airflow-providers-ftp==3.13.2
apache-airflow-providers-git==0.0.8
apache-airflow-providers-google==18.0.0
apache-airflow-providers-grpc==3.8.2
apache-airflow-providers-hashicorp==4.3.2
apache-airflow-providers-http==5.3.4
apache-airflow-providers-microsoft-azure==12.7.1
apache-airflow-providers-mysql==6.3.4
apache-airflow-providers-odbc==4.10.2
apache-airflow-providers-openlineage==2.7.1
apache-airflow-providers-postgres==6.3.0
apache-airflow-providers-redis==4.3.1
apache-airflow-providers-sendgrid==4.1.3
apache-airflow-providers-sftp==5.4.0
apache-airflow-providers-slack==9.3.0
apache-airflow-providers-smtp==2.2.1
apache-airflow-providers-snowflake==6.5.4
apache-airflow-providers-ssh==4.1.4
apache-airflow-providers-standard==1.8.0
apache-airflow-task-sdk==1.1.0
Jinja2==3.1.6
### Deployment
Docker-Compose
### Deployment details
N/A (fresh docker install)
### Anything else?
Happens 100% of the time.
Some possible clues:
The "create" task has this in its log:
```
Done. Returned value was: {'C': 1, 'B': 2, 'A': 3} source=airflow.task.operators.airflow.providers.standard.decorators.python._PythonDecoratedOperator loc=python.py:218
```
The order seems to be preserved up to this point. However, in its XCom pane, the `return_value` shows the dictionary with its keys sorted:
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with the minimal DAG using render_template_as_native_obj and trace the XCom return_value serialization between the create and render tasks. Compare the create-task log, XCom pane, and consuming-task output; done means the dictionary reaches the consuming task with its original insertion order.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100