stackabletech / stackabletech/airflow-operator
Investigate possible race conditions when gitsync-ing DAG submodules
Open
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 32
- Forks
- 6
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 13
Description
Description
As a user of Airflow I want to be sure that DAGs that use submodules are seamlessly updated and registered with Airflow via gitsync. This covers the following:
### Tasks
- [ ] Check that DAGs are consistent i.e. that the state when the DAG is started is the same despite subsequent syncs while the DAG is still running
- [ ] Check that "new" submodules are correctly loaded/registered
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the gitsync behavior around DAG submodules. Reproduce syncs while a DAG is running and add a new submodule, then verify that the running DAG remains consistent and that the new submodule's DAGs are loaded and registered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, rust
- Domain
- devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100