Unexpected WebSocketConnectionClosedException when streaming long running commands, missing pings?

从 kubernetes/stream/ws_client.py 开始，重点检查流连接以及 update/readline 路径，并使用提供的 sleep 命令和 _preload_content=False 示例复现空闲 300 秒后的断开连接。将其行为与 issue 中提到的 websocket-client 长连接指南进行比较；完成标准是空闲状态下的流式传输仍可用，或能在配置的限制内重新连接，且不会阻塞读取超时。

What happened (please include outputs or screenshots):
After exactly 300sec of idle, exec commands streamed with stream loose their connection, even though

• pod is still running,
• The process that was execd in container is still running (checked with kubectl exec POD_NAME ps aux)
• Network otherwise appears to be healthy.
• No warnings logged by kube-apiserver

Following stack trace is given

  File "kubernetes/stream/ws_client.py", line 141, in readline_stderr
    return self.readline_channel(STDERR_CHANNEL, timeout=timeout)
  File "kubernetes/stream/ws_client.py", line 104, in readline_channel
    self.update(timeout=(timeout - time.time() + start))
  File "kubernetes/stream/ws_client.py", line 192, in update
    op_code, frame = self.sock.recv_data_frame(True)
  File "websocket/_core.py", line 401, in recv_data_frame
    frame = self.recv_frame()
  File "websocket/_core.py", line 440, in recv_frame
    return self.frame_buffer.recv_frame()
  File "websocket/_abnf.py", line 338, in recv_frame
    self.recv_header()
  File "websocket/_abnf.py", line 294, in recv_header
    header = self.recv_strict(2)
  File "websocket/_abnf.py", line 373, in recv_strict
    bytes_ = self.recv(min(16384, shortage))
  File "websocket/_core.py", line 524, in _recv
    return recv(self.sock, bufsize)
  File "websocket/_socket.py", line 122, in recv
    raise WebSocketConnectionClosedException(
websocket._exceptions.WebSocketConnectionClosedException: Connection to remote host was lost. 

What you expected to happen:

• Connection is not lost
• If it is lost, connection is automatically reconnected (up to some configured timeout limit)
  Reading https://github.com/websocket-client/websocket-client#long-lived-connection, there is a description on how to use a built in way from the websocket library to do this. It appears that the stream command is not using this method.

How to reproduce it (as minimally and precisely as possible):
(Mostly inspired by https://github.com/kubernetes-client/python/blob/master/examples/pod_exec.py)

from kubernetes.stream import stream
...
...
s = stream(
    _client.connect_get_namespaced_pod_exec,
    _pod.metadata.name,
    _pod.metadata.namespace,
    command=["sleep", "100000"],           
    async_req=False,
    stderr=True,
    stdin=False,
    stdout=True,
    tty=False,
    _preload_content=False,
)

def tail_logs():
    if (ret_stdout := s.readline_stdout(timeout=1)) is not None:
        print(ret_stdout)

while (retcode := s.returncode) is None:
    tail_logs(s)
tail_logs(s)

Anything else we need to know?:
I've experimented by adding s.sock.ping() in each iteration of the tail_logs function and it appears to improve the situation but some times it still get stuck inside readline_stdout, despite the timeout=1 parameter being set. 🤨 It get stuck so long that ping never has chance to run within the next 300sec timeout.
Does one need to run ping in a separate thread for this to work? That's how it is done inside the WebSocketApp, as used by the long lived connection example from websocket library.
Can one use WebSocketapp together with stream or do i need to implement similar ping-feature myself when using stream?

I'm also curious where 300sec come from, is there a way to find this time through connection handshake or is it a setting in my cluster/azure?

Environment:

• Kubernetes version: Azure AKS v1.23.5
• Python version 3.9.7
• Python client version 23.6.0