redpanda-data / redpanda-data/benthos
Feature Request: Add a render or preview command to show template output with specific config values
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 571
- Forks
- 121
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 18
Description
Currently, there is no simple way to preview the fully rendered output of a Redpanda Connect template with a given set of config values. The existing workflows (rpk connect echo or rpk connect lint) either do not process templates or require writing test cases for every scenario.
Reference:
Slack thread: https://redpandacommunity.slack.com/archives/C075M18MAEA/p1749125364990089
Use Case:
As a developer, I want to quickly see what a template will render to with a specific config, for debugging and development purposes. This is especially useful when iterating on template logic or troubleshooting config issues, without having to write and maintain a test for every possible config variation.
command example:
rpk connect template render --template ./template.yaml --config ./config.yaml
template.yaml
name: kafka_input
type: input
fields:
- name: broker_address
type: string
kind: list
- name: topics
type: string
kind: list
- name: cert_path
type: string
- name: consumer_group
type: string
mapping: |
root.input = {
"kafka": {
"addresses" : this.broker_address,
"topics": this.topics,
"start_from_oldest": true,
"consumer_group": this.consumer_group,
"tls": {
"enabled": true,
"root_cas_file": "${this.cert_path}/ca.crt",
"client_certs": [
{
"cert_file": "${this.cert_path}/user.crt",
"key_file": "${this.cert_path}/user.key"
}
]
}
}
}
tests:
- name: kafka_test
config:
broker_address:
- broker-1
topics:
- topic-1
cert_path: kafka-secrets
consumer_group: app4134-composition-test
expected:
input:
kafka:
addresses:
- broker-1
topics:
- topic-1
consumer_group: app4134-composition-test
start_from_oldest: true
tls:
enabled: true
root_cas_file: kafka-secrets/ca.crt
client_certs:
- cert_file: kafka-secrets/user.crt
key_file: kafka-secrets/user.key
config.yaml
input:
kafka_input:
broker_address:
- broker-1
topics:
- topic-1
cert_path: demo-kafka-secret
consumer_group: benthos-local-test-4
pipeline:
processors:
- mapping: |
root.id = uuid_v4()
root.foo = this.inner.foo
root.body = this.outer`
expected output:
input:
kafka:
addresses:
- broker-1
topics:
- topic-1
start_from_oldest: true
consumer_group: benthos-local-test-4
tls:
enabled: true
root_cas_file: demo-kafka-secret/ca.crt
client_certs:
- cert_file: demo-kafka-secret/user.crt
key_file: demo-kafka-secret/user.key
pipeline:
processors:
- mapping: |
root.id = uuid_v4()
root.foo = this.inner.foo
root.body = this.outer
output:
stdout: {}`
Benefit:
This would greatly improve the developer experience for anyone building or debugging templates, and would make Redpanda Connect templating more approachable and productive.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by comparing the existing rpk connect echo and rpk connect lint commands with the proposed rpk connect template render entry point. Trace how template.yaml and config.yaml should be loaded and how the fully rendered configuration should be emitted; done means the example produces the expected output without requiring a test case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, developer-experience
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100