pyinfra-dev / pyinfra-dev/pyinfra
Nested `ProxyJump` is discarded: `connect()` overwrites the `sock` computed by `parse_config()`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6k
- Forks
- 548
- Avg merge
- 7d 17h
- Merged PRs (30d)
- 13
Description
Describe the bug
When a host's ProxyJump target is itself a host with its own ProxyJump (two levels of jumps), pyinfra
connects to that hop directly from the controller instead of through its jump chain, because
SSHClient.connect() overwrites the sock that parse_config() just computed for the hop:
src/pyinfra/connectors/sshuserclient/client.py:386—= self.parse_config(...)setscfg["sock"]to the hop's own jump chainsrc/pyinfra/connectors/sshuserclient/client.py:393—config.update(kwargs);kwargsstill containssock=<previous hop's channel, or None>(the hop loop passes it at:547), which clobbers the computed chain
Consequences:
- if the hop's
HostNameis127.0.0.1(typical when a jump host forwards a port on its own loopback) the connection silently lands on the controller's own sshd and fails with a confusingBadHostKeyException; - if the hop's
HostNameis not reachable from the controller you get the connection timeout of #1223 instead; - in both cases the real error is then masked by an unrelated internal error:
self._ssh_configis only assigned afterparse_config()returns (client.py:407), so theBadHostKeyExceptionhandler atpyinfra/connectors/ssh.py:279raisesAttributeError: 'SSHClient' object has no attribute '_ssh_config'.
Line numbers are from 3.x @ b3795cfa; the 3.10.0 wheel has the same code at client.py:214/221/235/371 and ssh.py:274.
To Reproduce
~/.ssh/config:
Host first
HostName 10.10.10.1
User network
Host hop # used as a ProxyJump target AND has its own ProxyJump
HostName 127.0.0.1 # any HostName reproduces; 127.0.0.1 makes the failure obvious
User core
ProxyJump network@10.10.10.1,core@10.20.0.1
Host target
HostName 127.0.0.1
User core
ProxyJump hop
# inventory.py
hosts = ["target"]
$ pyinfra inventory.py exec -- hostname
- Not reproducible with
@docker: this is the SSH connector'sssh_config/ProxyJumphandling and it fails during connect, before any operation runs. - The hop's host key is present in
~/.ssh/known_hosts, the controller's own sshd key is not, so the wrong-target connection is caught by paramiko's host key check. - Jump/target hosts: Ubuntu 22.04.5 LTS, stock OpenSSH server. Controller: see
pyinfra --supportbelow. - OpenSSH handles the identical config fine (
ssh target→ 4 handshakes, all host keys verified).
--> Loading config...
--> Loading inventory...
--> Connecting to hosts...
[pyinfra.connectors.ssh] Connecting to: target ({'allow_agent': True, 'look_for_keys': True, '_pyinfra_ssh_forward_agent': False, '_pyinfra_ssh_config_file': None, '_pyinfra_ssh_known_hosts_file': None, '_pyinfra_ssh_strict_host_key_checking': 'accept-new', '_pyinfra_ssh_paramiko_connect_kwargs': None, 'timeout': 10})
[pyinfra.connectors.sshuserclient.client] Loading SSH config: None
[pyinfra.connectors.sshuserclient.client] SSH ProxyJump through hop:22
[pyinfra.connectors.sshuserclient.client] SSH ProxyJump through 10.10.10.1:22
[pyinfra.connectors.sshuserclient.client] SSH ProxyJump through 10.20.0.1:22
Traceback (most recent call last):
File ".../site-packages/pyinfra/connectors/ssh.py", line 256, in _connect
self.client.connect(hostname, **kwargs)
File ".../site-packages/pyinfra/connectors/sshuserclient/client.py", line 214, in connect
) = self.parse_config(
File ".../site-packages/pyinfra/connectors/sshuserclient/client.py", line 371, in parse_config
c.connect(
File ".../site-packages/pyinfra/connectors/sshuserclient/client.py", line 244, in connect
super().connect(hostname, **config)
File ".../site-packages/paramiko/client.py", line 466, in connect
raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server '127.0.0.1' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIPBeN5aooxQmAaCqyxEEMxPEL96pDo2+/Muih//Rn9/o', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIAULf6x0958ZfU4rNxk8UEhmeQfUedn7LoDPHLOChDaO'
During handling of the above exception, another exception occurred:
AttributeError: 'SSHClient' object has no attribute '_ssh_config'
The "got" key is the controller's own sshd (fingerprint SHA256:OE10Z4rmA8/OfkBlkJqNUUNIhqeKjXeLJbtIC2JjSaI), the "expected" one is the hop's (SHA256:7zUPR4mNZzAzsvgkZreg2ExbHvl91nCB6KoxKiUq97o).
Expected behavior
- The controller connects to
hopthrough its own jump chain (first→10.20.0.1→127.0.0.1:22), exactly likessh targetdoes, and then totargetfrom there. - Host key verification is done against the hop's key, and if the connection does fail the reported error is the real one — not
AttributeError: 'SSHClient' object has no attribute '_ssh_config'.
Meta
pyinfra --support:
System: Linux
Platform: Linux-5.15.0-176-generic-x86_64-with-glibc2.35
Release: 5.15.0-176-generic
Machine: x86_64
pyinfra: v3.10.0
click: v8.5.0
distro: v1.9.0
gevent: v26.8.0
jinja2: v3.1.6
packaging: v26.3
paramiko: v4.0.0
pydantic: v2.13.5
python-dateutil: v2.9.0.post0
typeguard: v4.6.0
types-paramiko: v4.0.0.20260518
typing-extensions: v4.16.0
Executable: <venv>/bin/pyinfra
Python: 3.10.12 (CPython, GCC 11.4.0)
- Installed with
pip(viauv) from PyPI:pyinfra==3.10.0,paramiko==4.0.0. Also reproduced with the3.xtip installed from git (3.10.0.post36.dev0+b3795cfa, HEAD 2026-09-16). pyinfra-debug.logwas created (internal exception) — contents below.- The console output above was produced with
--debug;-vvadds nothing extra for this step. Paths and host names above are sanitised.
pyinfra-debug.log:
File ".../site-packages/pyinfra_cli/cli.py", line 290, in cli
_main(*args, **kwargs)
File ".../site-packages/pyinfra_cli/cli.py", line 448, in _main
connect_all(state)
File ".../site-packages/pyinfra/api/connect.py", line 37, in connect_all
greenlet.get()
File "src/gevent/greenlet.py", line 912, in gevent._gevent_cgreenlet.Greenlet.run
File ".../site-packages/pyinfra/api/host.py", line 390, in connect
self.connector.connect()
File ".../site-packages/pyinfra/connectors/ssh.py", line 226, in connect
return self._connect()
File ".../site-packages/pyinfra/connectors/ssh.py", line 274, in _connect
port = self.client._ssh_config.get("port", 22)
Traceback (most recent call last):
File ".../site-packages/pyinfra/connectors/ssh.py", line 256, in _connect
self.client.connect(hostname, **kwargs)
File ".../site-packages/pyinfra/connectors/sshuserclient/client.py", line 214, in connect
) = self.parse_config(
File ".../site-packages/pyinfra/connectors/sshuserclient/client.py", line 371, in parse_config
c.connect(
File ".../site-packages/pyinfra/connectors/sshuserclient/client.py", line 244, in connect
super().connect(hostname, **config)
File ".../site-packages/paramiko/client.py", line 466, in connect
raise BadHostKeyException(hostname, server_key, our_key)
paramiko.ssh_exception.BadHostKeyException: Host key for server '127.0.0.1' does not match: got 'AAAAC3NzaC1lZDI1NTE5AAAAIPBeN5aooxQmAaCqyxEEMxPEL96pDo2+/Muih//Rn9/o', expected 'AAAAC3NzaC1lZDI1NTE5AAAAIAULf6x0958ZfU4rNxk8UEhmeQfUedn7LoDPHLOChDaO'
During handling of the above exception, another exception occurred:
AttributeError: 'SSHClient' object has no attribute '_ssh_config'
Root cause
parse_config() starts from the caller's kwargs (cfg.update(initial_cfg or {})) and, in the proxyjump branch, sets cfg["sock"] to the chain it built. connect() then re-applies the original kwargs on top of the returned config, so the hop loop's sock=sock argument (the previous hop's channel, or None for the first hop) wins over the freshly built chain:
) = self.parse_config( # client.py:386 - sets cfg["sock"] for the hop's own ProxyJump
hostname,
kwargs,
...
)
self.set_missing_host_key_policy(missing_host_key_policy)
config.update(kwargs) # client.py:393 - clobbers config["sock"] with kwargs["sock"]
The hop loop itself is correct (client.py:547):
c = SSHClient()
c.connect(
hop_hostname,
_pyinfra_ssh_config_file=ssh_config_file,
sock=sock,
**hop_connect_kwargs,
)
Same root cause as #1223, which was closed via the "flatten your ~/.ssh/config" workaround; the code path is still present on 3.x and 4.x.
Separately, self._ssh_config = config (client.py:407) happens after parse_config() returns, while ssh.py:279 reads it in the BadHostKeyException handler — so an exception raised while connecting a hop inside parse_config() always degenerates into AttributeError and hides the actual failure.
Suggested fix
Preserve the sock computed by parse_config() (verified locally, fixes the repro above):
self.set_missing_host_key_policy(missing_host_key_policy)
+ proxyjump_sock = config.get("sock")
config.update(kwargs)
+ if proxyjump_sock is not None:
+ config["sock"] = proxyjump_sock
and stop masking the real error:
- port = self.client._ssh_config.get("port", 22)
+ port = getattr(self.client, "_ssh_config", {}).get("port", 22)
(or initialise _ssh_config before calling parse_config()).
There is also a test coverage gap: tests/test_connectors/test_sshuserclient.py covers single-level ProxyJump, ProxyJump none (#1445) and ConnectTimeout propagation, and always calls parse_config() directly, so the sock merge inside connect() and nested jumps are untested. A regression test could use a nested config, patch paramiko.SSHClient.connect (not pyinfra's, so the nested hop connects are still exercised) and assert that a hop with its own ProxyJump receives a non-None sock.
Workaround
Flatten the chain in ~/.ssh/config (same workaround as in #1223):
Host target
HostName 127.0.0.1
User core
ProxyJump network@10.10.10.1,core@10.20.0.1,core@hop
Two gotchas when flattening:
- don't spell the last hop
127.0.0.1if the target'sHostNameis also127.0.0.1— OpenSSH rejects it withjumphost loop via 127.0.0.1; uselocalhostinstead; - don't reference a jump-host alias: pyinfra uses
derive_shorthand()'s return value as the far-side dial target, so an alias gets resolved by name on the previous jump host (Secsh channel 0 open FAILED: Name or service not known).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with connect() and parse_config() in src/pyinfra/connectors/sshuserclient/client.py, especially the config merge and nested hop loop. Run tests/test_connectors/test_sshuserclient.py and add coverage for a nested ProxyJump while patching the underlying SSH client connections. Done means the nested hop receives its computed socket and connection failures retain their original error instead of becoming an AttributeError in src/pyinfra/connectors/ssh.py.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100