kubernetes-client / kubernetes-client/python

When using follow=True option to read pod logs, the response stream sometimes returns many log lines at once

Open
#2,426 3 comments 0 reactions 0 assignees View on GitHub
help wanted kind/bug
Dominant language
Python
Stars
7.7k
Forks
3.5k
Avg merge
1d 14h
Merged PRs (30d)
18

Description

**What happened (please include outputs or screenshots)**:
I'm using `read_namespaced_pod_log`, setting `follow=True` and `_preload_content=False`:
```
response = client.read_namespaced_pod_log(follow=True, ...)
for log in response.stream():
...
```
We expect each line to be valid JSON. However something changed recently, so that sometimes, a single item returned from the `stream` contains multiple log lines, so that we fail to parse it. When I run the exact same command but without setting `follow=True`, and call `splitlines()` on the whole `response`, each line is separated correctly.

With `Follow=True`: (removing a lot of details from these logs, keeping the structure and newlines)
```
> log_lines = get_logs_for_pods()
(Pdb) !l=next(log_lines)
(Pdb) p l
b'{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}\n{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}\n{"level":"info","ts":...,"caller":"...","msg":"...","rel":"snip","tr":{"tid":"...","sid":"...","op":"..."}}\n'
```

Without:
```
{"level":"info","ts":...,"caller":"...","msg":"...","rel":"...","tr":{"tid":"...","sid":"...","op":"..."}}
```

**What you expected to happen**:
Each item from the stream would be a single log line.

I am not 100% sure if it's expected for the SDK to behave this way or not, based on the documentation. I can split the lines myself, but it would be a lot more intuitive if each item were a single log line.

**How to reproduce it (as minimally and precisely as possible)**:

**Anything else we need to know?**:

**Environment**:
- Kubernetes version (`kubectl version`):
```
Client Version: v1.30.2
Server Version: v1.32.4-gke.1698000
```
- OS (e.g., MacOS 10.13.6): Sequoia 15.5
- Python version (`python --version`): python 3.8.13
- Python client version (`pip list | grep kubernetes`): 20.13.0

Contributor guide

Open the contributing guide

Research direction

Start at the Python client's read_namespaced_pod_log entry point and follow how response.stream() handles follow=True compared with non-following responses. Reproduce the reported batching with the versions in the issue, then establish whether stream items should contain single lines; done means the behavior is clarified or corrected and covered by a focused regression check.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, python
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.