calculate_dagrun_date_fields doesn't understand how it is called
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Body
It has an argument `last_automated_run` but in reality, what it is given is not always the latest run.
This is because sometimes it is called, in the scheduler, with whatever run the scheduler happens to be dealing with -- which might not be the latest run.
And even if something _thinks_ it is passing the latest run, due to airflow being a distributed system, something else could create a new run in the meantime, making it not the latest run anymore.
For one thing, this behavior sometimes causes airflow to set the next_dagrun field back in time.
We should probably at least rename this field. But it might also be worth looking at how it's called and possibly making some adjustments to the logic.
### Committer
- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Contributor guide
Research direction
Start at calculate_dagrun_date_fields and the scheduler call sites described in the issue. Trace whether last_automated_run is actually the latest run, then define tests covering next_dagrun moving backward; done means the argument meaning and scheduler behavior are consistent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100