stackabletech / stackabletech/spark-k8s-operator
dynamically create ingress for running sparkapplication to access spark-ui outside of the k8s cluster
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 72
- Forks
- 4
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 15
Description
when submitting a sparkapplication resource the driver will expose port 4040 where we can access the spark-ui. unfortunately the ui is available only within the cluster, not outside of it by using a webbrowser on a users pc.
of course it's possible to create an ingress by myself for every spark application I'm submitting but since sparkapplications are ephemeral after a couple of weeks there will be lots of dead ingresses in the cluster because the spark application they belong to has terminated already.
i think the operator should create an ingress / route whenever a sparkapplication is submitted. possible configuration option would be a value set in the operator itself. i'd prefer that rather than setting the ingress configs with every application
@sbernauer feel free to add
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the operator's SparkApplication submission and reconciliation entry points. Determine how a configurable ingress or route would be created for port 4040 and removed when the SparkApplication terminates. Done means the UI is reachable outside the cluster without leaving dead ingress resources.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, rust, spark
- Domain
- infrastructure, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100