opensearch-project / opensearch-project/data-prepper
Enhance convert_entry_type processor to convert multiple fields in an event
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
When a given event is a list of objects with some attributes, I would like to apply convert_entry_type processor on a field in all of these objects. For example, if my event looks like below and I would like to convert all key1 attributes using convert_entry_type processor
{
[
{ "key1": "value1", "key2": "value2"},
{ "key1": "value3", "key2": "value4"},
{ "key1": "value5", "key2": "value6"}
]
}
Describe the solution you'd like
Similar to https://github.com/opensearch-project/data-prepper/issues/2853 having an optional "iterate-on" functionality on convert_entry_type processor will help apply the conversion on multiple fields of a given event.
Describe alternatives you've considered (Optional)
Support for providing Json Path instead of Json Pointer that points to a specific field.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the convert_entry_type processor and the related issue #2853, which describes similar iterate-on functionality. Confirm how the processor currently targets one field and define the desired behavior for the list-of-objects example. Done means an optional iterate-on capability converts the selected field in every object, with coverage for the example behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100