scheduler: Ability to create a DAG where it is repeatable given start date, end date and CRON
- Dominant language
- Go
- Stars
- 4k
- Forks
- 332
- Avg merge
- 19h 26m
- Merged PRs (30d)
- 133
Description
For example, if have I have
```
cron: 5 * * * *
start_date: 2024-10-10
end_date: 2024-12-10
```
This DAG will generate tasks that run every 5 minutes, and generate a variable (like $CRON_DATE) that can be passed to the underlying shell commands.
This is useful when you have historical data that is divided by hours, and you want to do a bunch of processing on them.
Contributor guide
Research direction
Start by locating the scheduler path that parses CRON expressions and DAG dates, then trace how scheduled values reach underlying shell commands. Done means a DAG can repeat between the supplied start and end dates, runs at the requested CRON times, and exposes a date variable to those commands.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100