stackabletech / stackabletech/airflow-operator
Fix: Remove the possibility to add multiple git-sync resources as DAG backends
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 32
- Forks
- 6
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 13
Description
Context
We have tried to implement multiple git-sync resources into one DAGS_CORE_FOLDER which was successful.
However when the feature was completed we noticed problems with this approach like the local dag path bleeds into DAG logic. After investigating we found reasons to rather not do it: https://github.com/stackabletech/airflow-operator/issues/721#issuecomment-4038282638
This now leads to the decision to not expose a possibility to add multiple dags in the Airflow CR.
What needs to happen
We want to remove the possibility to
clusterConfig:
credentialsSecret: test-airflow-credentials
dagsGitSync:
- repo: https://github.com/a-pretty-repo
branch: feat/beautiful-branch
gitFolder: "a-passably-folder"
- repo: https://github.com/a-grave-repo
credentialsSecret: git-credentials
gitSyncConf:
# supply some config to check that safe.directory is correctly set
--git-config: http.sslVerify:false
gitFolder: "a-well-formed-folder"
and rather have something like:
clusterConfig:
credentialsSecret: test-airflow-credentials
dagsGitSync:
repo: https://github.com/the-master-repo
branch: feat/mighty-branch
gitFolder: "all-my-magic"
as leaving this option open suggest a incomplete feature and might raise expectation on us to implement it.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source files or tests are named in the issue. Start by locating the Airflow CR definition and all references to clusterConfig.dagsGitSync; done means the configuration accepts one git-sync resource rather than a list and the existing behavior remains covered by the project's checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, rust
- Domain
- backend, infrastructure
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100