opensearch-project / opensearch-project/data-prepper
HTTP source does not support non-array JSON payload
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Data Prepper returns an error when receiving a JSON payload that is not wrapped in an array.
For example: curl -d '{"key":"value"}' http://localhost:2021/log/ingest returns Bad request data format. Needs to be json array..
Describe the solution you'd like
Allow HTTP source to handle both JSON array and JSON object payloads.
Describe alternatives you've considered (Optional)
We would have to add an additional service layer that can handle both payload formats (like Fluent-Bit or Logstash) before Data Prepper.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the HTTP source handling the /log/ingest entry point and trace where JSON input is required to be an array. Use the reported curl payloads as the reproduction cases, then verify that both array and object payloads are accepted without the current format error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100