apache / apache/airflow

xcom include_prior_dates=True returns "None" instead of xcom

Open
#70,173 0 comments 0 reactions 0 assignees View on GitHub
area:core kind:bug needs-triage
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 9h
Merged PRs (30d)
472

Description

### Under which category would you file this issue?

Airflow Core/Task SDK

### Apache Airflow version

3.2.2, 3.3.0

### What happened and how to reproduce it?

Try running this simple dag pair for cross dag xcom transfer. The cross_dag_xcom_downstream dag correctly receives the xcom value at airflow v3.1.7 but receives None in airflow v3.2x and v3.3.0. The cross_dag_xcom_upstream correctly creates an xcom entry its just that the cross_dag_xcom_downstream is not able to access it.

```
from airflow.sdk import dag, task

@dag()
def cross_dag_xcom_upstream():
@task
def push_to_xcom():
return "Hello!"

push_to_xcom()

cross_dag_xcom_upstream()

@dag()
def cross_dag_xcom_downstream():
@task
def pull_from_xcom(**context):
print("The context task instance is", context["task_instance"])

my_data = context["task_instance"].xcom_pull(
dag_id="cross_dag_xcom_upstream",
task_ids="push_to_xcom",
key="return_value",
include_prior_dates=True,
)
return my_data

pull_from_xcom()

cross_dag_xcom_downstream()
```

### What you think should happen instead?

The downstream dag should receive the xcom.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

Start by running the two DAGs shown in the issue on Airflow 3.2.2 or 3.3.0 and compare the TaskInstance.xcom_pull call with Airflow 3.1.7. Trace the airflow.sdk task and cross-DAG XCom lookup, focusing on include_prior_dates=True. Done means the downstream task receives "Hello!" and the regression is covered by a reproducible check.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.