apache / apache/airflow

Dagrun "skipped" for timezone aware DAGs with schedules affected by daylight saving time

Open
#38,098 2 comments 1 reaction 0 assignees View on GitHub
area:core area:scheduler kind:documentation
Dominant language
Python
Stars
46.9k
Forks
17.8k
Avg merge
2d 7h
Merged PRs (30d)
484

Description

### What do you see as an issue?

The issue is that users are unaware of scheduling behaviour between transition of when [daylight saving time](https://en.wikipedia.org/wiki/Daylight_saving_time) is observed and when it is not.

For daylight time saving observing timezones, at the cusp of when switching between when daylight saving time is **NOT** observed to when it is, there is an hour that does not exist. For example, 2 AM Central Time ("America/Chicago") does not exist on March 10th 2024.

* `2024-03-10 7:59 UTC is 1:59 AM CT (CST)`
* `2024-03-10 8:00 UTC is 3:00 AM CT (CDT)`
* `2024-03-10 8:01 UTC is 3:01 AM CT (CDT)`

For daylight time saving observing timezones, at the cusp of when switching between when daylight saving time is observed to when it is **NOT**, there is an hour that is repeated. For example, 1 AM Central Time ("America/Chicago") happens twice on November 3rd 2024.

* `2024-11-03 6:59 UTC is 1:59 AM CT (CDT)`
* `2024-11-03 7:00 UTC is 1:00 AM CT (CDT)`
* `2024-11-03 7:01 UTC is 1:01 AM CT (CST)`
* `2024-11-03 7:59 UTC is 1:59 AM CT (CST)`
* `2024-11-03 8:00 UTC is 2:00 AM CT (CST)`
* `2024-11-03 8:01 UTC is 2:01 AM CT (CST)`

### Solving the problem

A description of the effects of these impacted schedule intervals in terms of dagrun creation as that is what drives task scheduling. Without a dagrun, there are no tasks.

### Anything else

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

Contributor guide

Open the contributing guide

Research direction

No file, test, or entry point is named. Start by reproducing the scheduler's dagrun creation around the documented America/Chicago daylight-saving transitions and trace how nonexistent and repeated local times are handled. Done requires an agreed, tested description of the expected dagrun behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.