opensearch-project / opensearch-project/data-prepper
Supporting iterate_on for write_json
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
When using write_json on a key with an array value, I would like to be able to convert each element in the array to its json string representation. Currently if an array is passed to write_json it will generate the json string representation of the array.
For example currently
"key1": [
{"a":"b", "c":"d"}
]
with the processor
- write_json:
- source: key1
target: some_key
outputs
"some_key": "[{\"a\":\"b\",\"c\":\"d\"}]"
But with iterate_on it would look something like this
"some_key": [
"{\"a\":\"b\",\"c\":\"d\"}",
]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered (Optional)
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the write_json processor implementation and its tests, then compare them with the existing iterate_on behavior in other processors. Confirm how array inputs are currently serialized and define tests showing that iterate_on produces one JSON string per array element while preserving current behavior without it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100