redpanda-data / redpanda-data/connect

Logger destination may be inconsistent - suggestion for a new Logger configuration field

Open
#2,156 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement observability ux
Dominant language
Go
Stars
8.8k
Forks
969
Avg merge
1d 13h
Merged PRs (30d)
64

Description

The documentation specifies that

Benthos logging prints to stdout (or stderr if your output is stdout)

Unfortunately this doesn't work if the output uses a switch.
Benthos prints to stdout in this case, even if the selected case condition is stdout output.

To reproduce:

input:
  generate:
    mapping: |
      root.id = count("foo_counter1")
    interval: 0s
    count: 3

output:
  switch:
    cases:
      - check: false
        output:
          drop: {}
      - output:
          stdout:
            codec: lines

This will print to stdout, but stderr is expected.

Also, considering the Logger doc:

Use these configuration options to change both the logging formats as well as the destination of logs.

A way of solving this issue could be to "force" the destination with a new field (destination? write_to?) in the Logger configuration.

Without too much reflection (especially about the file output), it could look to something like this:

logger:
  level: INFO
  format: logfmt
  write_to: stderr # stdout | stderr | file?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided YAML switch-output reproduction and read the Logger configuration documentation alongside the output-switch behavior. Determine how an explicit logger destination should interact with stdout, stderr, and file output; done means the documented configuration resolves the inconsistency and the reproduction matches the selected destination.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.