redpanda-data / redpanda-data/connect
Different behavior in JSON environment variable parsing during testing vs. linting
Open
Nobody has claimed this yet.
needs investigation
- Dominant language
- Go
- Stars
- 8.8k
- Forks
- 969
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 64
Description
Given the files:
.env
MY_JSON="[{ \"id\": 1234 }]"
benthos_benthos_test.yml
tests:
- name: dummy
target_processors: "/input/processors"
environment:
MY_JSON: '[{ \"id\": 1234 }]'
input_batch:
- content: ""
output_batches:
- - json_equals: [{ "id": 1234 }]
benthos.yml
input:
generate:
count: 1
interval: ""
mapping: |
root = ""
processors:
- mapping: |
root = "${MY_JSON}".parse_json()
Scenario 1
When I run test
$ benthos test ./...
Test 'benthos.yml' succeeded
But when I run lint
$ benthos -e .env lint ./... && echo succeeded
benthos.yml(9,29) line 1 char 13: expected line break
|
1 | root = "[{ "id": 1234 }]".parse_json()
| ^---
Scenario 2
Changing benthos.yml to
input:
generate:
count: 1
interval: ""
mapping: |
root = ""
processors:
- mapping: |
root = ${MY_JSON}
When I run test:
$ benthos test ./...
Failed to execute test target 'benthos.yml': test case 0 failed: failed to initialise processors '/input/processors': failed to initialise processor index '0': failed to init processor <no label>: failed to parse bloblang mapping '': expected query, got: \"id\
But when I run lint:
$ benthos -e .env lint ./... && echo succeeded
succeeded
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the two scenarios using .env, benthos_benthos_test.yml, and benthos.yml, running benthos test ./... and benthos -e .env lint ./.... Compare how environment variables are expanded and parsed in testing versus linting; done should mean the discrepancy is resolved or its intended behavior is explicitly established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- testing, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100