redpanda-data / redpanda-data/connect

Allow the StreamBuilder.AsYaml to have options to disable scrubbing

Open
#1,939 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ash pls configuration enhancement ux
Dominant language
Go
Stars
8.8k
Forks
969
Avg merge
1d 13h
Merged PRs (30d)
64

Description

TL;DR - Scrubbing control is broken as Spec.SanitiseYAML does not take the SanitiseConfig into account for scrubbing

When you want to output the result of your StreamBuilder, you can call the AsYaml function to return a scrubbed version of your configuration. This is great because it formats the configuration file in a more readable way than what we would get doing the yaml Marshalling ourselves. However, there does not seem to be a way to disable the scrubbing.

Underneath a dedicated scrubber is used (defined on config field level) to scrub the actual value in case the field contains a secret. A call to AsYaml will create a SanitiseConfig containing various options, one being to control whether scrubbing should happen or not. However, this config is not taken into account by the scrubber defined on the field.

Approach:

  • Guard the scrubbing behavior in the Spec.SanitiseYAML based on SanitiseConfig.ScrubSecrets
  • Make sure the NewSanitiseConfig is enabling scrubbing by default
  • Allow GenerationOptions to be passed to the StreamBuilder.AsYaml() call
  • Create a WithScrubbingDisabledoption to be passed to the StreamBuilder.AsYaml() call

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with StreamBuilder.AsYaml and Spec.SanitiseYAML, then trace SanitiseConfig, NewSanitiseConfig, and GenerationOptions. Done means ScrubSecrets controls field-level scrubbing, scrubbing remains enabled by default, and AsYaml accepts a WithScrubbingDisabled option.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
stream-processing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.