redpanda-data / redpanda-data/connect

sftp output does not retry after a failed transfer operation. (Connection failed)

Open
#3,584 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
8.8k
Forks
969
Avg merge
1d 13h
Merged PRs (30d)
64

Description

Version 4.61.0:

It seems that when there is an issue with the sftp, say a permissions issue on overwriting a file, the sftp output will never reconnect to the sftp server. First there is a permission denied issue overwriting a file on the remote:

time="2025-08-12T17:45:56Z" level=error msg="Failed to send message to sftp: failed to open remote file: permission denied" @service=redpanda-connect label="" path=root.output.broker.outputs.0 stream=sftp

Then after the failure I start seeing these messages in the log:

time="2025-08-12T17:45:57Z" level=error msg="Failed to send message to sftp: failed to create SFTP client: ssh: rejected: connect failed (open failed)" @service=redpanda-connect label="" path=root.output.broker.outputs.0 stream=sftp

And on and on forever until this morning:

time="2025-08-13T16:21:14Z" level=error msg="Failed to send message to sftp: failed to create SFTP client: ssh: rejected: connect failed (open failed)" @service=redpanda-connect label="" path=root.output.broker.outputs.0 stream=sftp

If I restart my redpanda-connect pod, transfers resume normally. I believe this is a bug with the reuse of the ssh connection in the sftp output.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the SFTP output implementation and tracing how it reuses the SSH connection after a failed remote file operation. Reproduce the permission-denied transfer failure, then verify that the output reconnects and resumes transfers without restarting the redpanda-connect pod.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, stream-processing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.