kubeflow / kubeflow/spark-operator
Fix Spark Connect container argumenets construction
- Dominant language
- Python
- Stars
- 3.2k
- Forks
- 1.5k
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 13
Description
### What happened?
Currently Spark Connect controller constructs container command and args with
```
container.Command = []string{"bash", "-c"}
args, err := buildStartConnectServerArgs(conn)
if err != nil {
return fmt.Errorf("failed to build spark connection args: %v", err)
}
container.Args = []string{strings.Join(args, " ")}
```
This is not ideal because everything is wrapped in `bash -c` which means all values passed in with the arg field are subject to shell interpretation.
We need to follow the way Spark Application Controller does it here: https://github.com/jayanth86/spark-operator/blob/00ba077967057560ad34949906d5ff4846659ce2/internal/controller/sparkapplication/submission.go#L71-L94
### Reproduction Code
_No response_
### Expected behavior
_No response_
### Actual behavior
_No response_
### Environment & Versions
- Kubernetes Version:
- Spark Operator Version:
- Apache Spark Version:
### Additional context
_No response_
### Impacted by this bug?
Give it a 👍 We prioritize the issues with most 👍
Contributor guide
Research direction
Locate the Spark Connect controller code that calls buildStartConnectServerArgs and compare its container command and argument construction with internal/controller/sparkapplication/submission.go from the referenced commit. Confirm that arguments are passed without bash -c or shell joining, and run the relevant controller tests to verify Spark Connect startup still works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100