Partitioned DAG has two Pending Dag Runs with the same partition key
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.3.0
### What happened and how to reproduce it?
DAG was not triggered due to 2 Pending Dag runs still waiting on different asset events.
Both however have the same Mapped Partition Key and should be considered as one.
Additionally clicking on the received asset events an empty tooltip is shown:
Records in asset_partition_dag_run show two records made with one ms apart. Both are identical except for the partition_date.
One record has it filled in, the other is empty:
### What you think should happen instead?
DAG should be triggered when all 4 events were received.
### Operating System
Ubuntu 22.04.5 LTS
### Deployment
Virtualenv installation
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by reproducing the partitioned DAG behavior described for Apache Airflow 3.3.0 and inspect the asset_partition_dag_run records, especially the duplicate rows that differ only in partition_date. Trace how the pending runs and received asset events are represented, including the empty event tooltip. Done means identical mapped partition keys are treated as one run, the DAG triggers after all four events, and the tooltip displays the event details.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100