kubeflow / kubeflow/spark-operator

Relevance of spark version in SparkApplication and SparkConnect CRD

Open
#3,149 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.2k
Forks
1.5k
Avg merge
5d 10h
Merged PRs (30d)
13

Description

The goal of the issue is to identify if setting the spark version field in the SparkApplication and SparkConnect CRD to mandatory is ideal - given that upstream Spark runtime does not have a flag for the version and the image itself is set to be optional on the CRD. The image is what happens to be of actual operational consequence - not the version.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the SparkApplication and SparkConnect CRD definitions and how their spark version and image fields are declared. Compare those fields with the upstream Spark runtime behavior described in the issue, then document whether version should remain mandatory and what CRD change, if any, would be required.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, spark
Domain
api, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.