stackabletech / stackabletech/spark-k8s-operator
sparkImage.pullPolicy does not propagate to driver
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 72
- Forks
- 4
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 15
Description
Affected Stackable version
26.7
Affected Apache Spark-on-Kubernetes version
4.x
Current and expected behavior
When setting sparkImage.pullPolicy to Always, this is configured for the submit pod, but at least the driver keeps the default setting. Expected would be that we use the pullPolicy (and other sparkImage settings) on driver and executor pods.
Workaround
spec:
sparkConf:
spark.kubernetes.container.image.pullPolicy: "Always"
Possible solution
No response
Additional context
No response
Environment
No response
Would you like to work on fixing this bug?
No response
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how sparkImage.pullPolicy is applied to the submit pod, then compare the configuration passed to the driver and executor pods. Reproduce the issue with sparkImage.pullPolicy set to Always and verify that the driver and executor pod settings match the requested sparkImage configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, rust, spark
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100