kubeflow / kubeflow/sdk

Fix in-cluster Spark Connect URL: use .svc instead of .svc.cluster.local

Open Beginner friendly
#490 3 comments 0 reactions 0 assignees View on GitHub
area/spark kind/bug
Dominant language
Python
Stars
148
Forks
262
Avg merge
1d 2h
Merged PRs (30d)
1

Description

### What happened?

When `SparkClient` runs **in-cluster** (for example from a notebook pod or an E2E job inside the cluster), `KubernetesBackend` builds the Spark Connect URL via `build_service_url()` in [kubeflow/spark/backends/kubernetes/utils.py](https://github.com/kubeflow/sdk/blob/0.4.0/kubeflow/spark/backends/kubernetes/utils.py#L84):
```python
return f"sc://{service}.{info.namespace}.svc.cluster.local:{constants.SPARK_CONNECT_PORT}"
```
That produces URLs like:
`sc://my-session-svc.spark.svc.cluster.local:15002`

This does not match the documented/expected Kubernetes cluster DNS short form (`..svc`) and diverges from KEP-107 examples (`sc://spark-cluster.spark-system.svc:15002`).

### What did you expect to happen?

In-cluster URLs should use the standard cluster DNS suffix:
```python
return f"sc://{service}.{info.namespace}.svc:{constants.SPARK_CONNECT_PORT}"
```
Example:

`sc://my-session-svc.spark.svc:15002`

**Expected behavior**
- In-cluster `SparkClient.connect()` (create mode, no `base_url`) resolves the session service via `-svc..svc:`.
- `build_service_url()` and unit tests align with this format.
- Comments in e2e/hack docs that still mention `svc.cluster.local` are updated for consistency.
**Suggested scope**
- Change `build_service_url()` in [kubeflow/spark/backends/kubernetes/utils.py](https://github.com/kubeflow/sdk/blob/0.4.0/kubeflow/spark/backends/kubernetes/utils.py#L84):
- Update [kubeflow/spark/backends/kubernetes/utils_test.py](https://github.com/kubeflow/sdk/blob/0.4.0/kubeflow/spark/backends/kubernetes/utils_test.py) (and any backend tests asserting `svc.cluster.local`).
- Refresh e2e/doc comments if they reference the old hostname.

### Environment

Kubernetes version:
```bash
$ kubectl version
Client Version: v1.35.3
Kustomize Version: v5.7.1
Server Version: v1.33.0
```
Kubeflow Trainer version:
```bash
$ kubectl get pods -n kubeflow -l app.kubernetes.io/name=trainer -o jsonpath="{.items[*].spec.containers[*].image}"

```
Kubeflow Python SDK version:
```bash
$ pip show kubeflow
0.4.0
```

### Impacted by this bug?

Give it a 👍 We prioritize the issues with most 👍

Contributor guide

Open the contributing guide

Research direction

Start with build_service_url() in kubeflow/spark/backends/kubernetes/utils.py and run kubeflow/spark/backends/kubernetes/utils_test.py. Search backend tests and e2e or hack comments for svc.cluster.local references. Done means in-cluster URLs and their tests use the documented .svc form consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
backend, networking
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.