ssh-agent loses connection to clients

"OpenSSH for Windows" version
8.1.0.1

Server OperatingSystem
Windows 10 Enterprise

Client OperatingSystem
Linux docker container (maybe also native linux not sure)
debian 11, ssh -V: OpenSSH_8.4p1 Debian-5, OpenSSL 1.1.1k 25 Mar 2021

What is failing
I am using VSCode with Remote Container Plugin on a remote docker host (name: xpw).
I when I start the container, ssh (in git pull etc) works via public keys resolved by ssh agent.
I can see the keys from windows with ssh-add -l. The are forwarded: Windows -> xpw -> docker@xpw

FYI: there is also a VPN between Windows and xpw!

After some time, the ssh-agent connection crashes. I initially thought it was related to powersave mode, but i just got it without any break
VSCode is still connected, but git pull does not work anymore. It simply blocks.

ssh-add -l also blocks.
Also, ssh to another machine that initially works now blocks:

build-user@xpw:/workspace$ ssh -v xpw
OpenSSH_8.4p1 Debian-5, OpenSSL 1.1.1k  25 Mar 2021
debug1: Reading configuration data /home/build-user/.ssh/config
debug1: /home/build-user/.ssh/config line 2: Applying options for *
debug1: /home/build-user/.ssh/config line 7: Applying options for *
debug1: /home/build-user/.ssh/config line 10: Applying options for xpw
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket "/home/build-user/.ssh/.socket_user@xpw-22" does not exist
debug1: Connecting to xpw [127.0.1.1] port 22.
debug1: Connection established.
debug1: identity file /home/build-user/.ssh/public/user_at_xpw.pub type 0
debug1: identity file /home/build-user/.ssh/public/user_at_xpw.pub-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.4p1 Debian-5
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.3
debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 pat OpenSSH* compat 0x04000000
debug1: Authenticating to xpw:22 as 'user'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:xxxxxxxxxxxxxxxxx
debug1: Host 'xpw' is known and matches the ECDSA host key.
debug1: Found key in /home/build-user/.ssh/known_hosts:20
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks

Restarting ssh-agent on windows does not work.
Then, ssh-add fails instantly:

build-user@xpw:/workspace/platform_rts$ ssh-add -l
error fetching identities: communication with agent failed

Reopening the container always fixes the issue.

Expected output
ssh-agent connection should not be lost.
ssh, git pull and ssh-add -l should always work.

Actual output
s.a.

The report names no source file or test. Start by reproducing the forwarded ssh-agent path from Windows through xpw into the Docker container, then inspect the agent communication and logs when ssh-add -l blocks or fails. Done means the forwarded agent remains usable and ssh, git pull, and ssh-add -l continue to work without reopening the container.