Spark log retrieval returns an extra empty line for empty or newline-terminated output
- Dominant language
- Python
- Stars
- 148
- Forks
- 262
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 1
Description
### What happened?
While reviewing the remaining Spark unit-test coverage in #609, I noticed that `read_pod_logs()` splits non-streaming log responses using `logs.split("\n")`.
This produces unexpected results:
```python
"".split("\n") == [""]
"line\n".split("\n") == ["line", ""]
"line 1\r\nline 2".split("\n") == ["line 1\r", "line 2"]
```
As a result, empty logs are represented as one blank line, newline-terminated logs contain an extra empty entry, and CRLF responses retain the carriage-return character.
`read_pod_logs()` is shared by both Spark Connect session logs and Spark batch-job logs.
Relevant code: `kubeflow/spark/backends/kubernetes/utils.py::read_pod_logs`
### What did you expect to happen?
Non-streaming log output should be returned as logical lines:
- Empty output should return no lines.
- A trailing newline should not create an additional empty entry.
- CRLF line endings should be handled correctly.
- Genuine blank lines inside the output should remain preserved.
Using `splitlines()` for the non-streaming response would provide this behavior. Unit tests should cover empty output, trailing newlines, CRLF output, and interior blank lines.
### Environment
Kubernetes version:
Not applicable. The issue is in client-side parsing of the string returned by the Kubernetes Python client and does not require a running cluster to reproduce.
Kubeflow Trainer version:
Not applicable.
Kubeflow Python SDK version:
Current `main` branch, observed at commit `8ecc659abbb13d089ee1046f5d967d5ad5a4da11`.
Relevant file: `kubeflow/spark/backends/kubernetes/utils.py`
Contributor guide
Assessment
This issue has not been assessed yet.