kubernetes-client / kubernetes-client/python
When using follow=True option to read pod logs, the response stream sometimes returns many log lines at once
- Dominant language
- Python
- Stars
- 7.7k
- Forks
- 3.5k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 18
Description
**What happened (please include outputs or screenshots)**:
I'm using `read_namespaced_pod_log`, setting `follow=True` and `_preload_content=False`:
```
response = client.read_namespaced_pod_log(follow=True, ...)
for log in response.stream():
...
```
We expect each line to be valid JSON. However something changed recently, so that sometimes, a single item returned from the `stream` contains multiple log lines, so that we fail to parse it. When I run the exact same command but without setting `follow=True`, and call `splitlines()` on the whole `response`, each line is separated correctly.
With `Follow=True`: (removing a lot of details from these logs, keeping the structure and newlines)
```
> log_lines = get_logs_for_pods()
(Pdb) !l=next(log_lines)
(Pdb) p l
b'{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}\n{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}\n{"level":"info","ts":...,"caller":"...","msg":"...","rel":"snip","tr":{"tid":"...","sid":"...","op":"..."}}\n'
```
Without:
```
{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}
```
**What you expected to happen**:
Each item from the stream would be a single log line.
I am not 100% sure if it's expected for the SDK to behave this way or not, based on the documentation. I can split the lines myself, but it would be a lot more intuitive if each item were a single log line.
**How to reproduce it (as minimally and precisely as possible)**:
**Anything else we need to know?**:
**Environment**:
- Kubernetes version (`kubectl version`):
```
Client Version: v1.30.2
Server Version: v1.32.4-gke.1698000
```
- OS (e.g., MacOS 10.13.6): Sequoia 15.5
- Python version (`python --version`): python 3.8.13
- Python client version (`pip list | grep kubernetes`): 20.13.0
Contributor guide
Research direction
Start at the Python client's read_namespaced_pod_log entry point and follow how response.stream() handles follow=True compared with non-following responses. Reproduce the reported batching with the versions in the issue, then establish whether stream items should contain single lines; done means the behavior is clarified or corrected and covered by a focused regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100