redpanda-data / redpanda-data/connect

Unconnected inproc output causes pipeline to stall

Open
#1,780 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

annoying needs investigation outputs
Dominant language
Go
Stars
8.8k
Forks
969
Avg merge
1d 13h
Merged PRs (30d)
64

Description

I think there's an issue in Benthos - well not sure it's an issue - where an unconnected inproc output fails quietly. As in, it doesn't even trigger fallback to go to next output. Minimal repro:

input:
  generate:
    mapping: |
      root = nanoid()
    interval: 3s

output:
  fallback:
    - inproc: nonexistent
    - stdout: {}

Run this with streams mode.

Expected behaviour: there's an immediate error at the inproc output that triggers a fallback to stdout.
Actual behaviour: the benthos pipeline stalls indefintely due to backpressure while it waits for in-flight messages to be processed by inproc.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the minimal YAML reproduction in the issue and run it in streams mode. Trace the inproc output and fallback behavior for the nonexistent target, then verify that the pipeline reports an immediate error and falls back to stdout instead of stalling under backpressure.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
stream-processing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.