stackabletech / stackabletech/spark-k8s-operator
Support syncing PySpark applications from Git
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 72
- Forks
- 4
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 15
Description
In case you have multiple Python files that form an application it is inconvenient to put everything in a ConfigMap.
It would be nice if you could use the same mechanism as implemented in Airflow using gitsync.
Related to https://github.com/stackabletech/opa-operator/issues/504, which implements the same thing for OPA regorules.
### Tasks
- [ ] Try to move commons code into operator-rs
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the related opa-operator issue #504 and its gitsync implementation, then review how this operator currently places multiple Python files in a ConfigMap. The operator-rs task indicates that shared code may need to move there; done means PySpark applications can be synced from Git instead of requiring all files in a ConfigMap.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, kubernetes, python, rust
- Domain
- data-engineering, devops
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100