Graylog2 / Graylog2/graylog2-server
Input type JSON path from HTTP API doesn't store valid json when specifying JSON path of data to extract
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
When using the input type `JSON path from HTTP API`, you are required to provide a value for `JSON path of data to extract`.
However, any value placed in this field other than `*` returns invalid json that cannot be parsed.
## Expected Behavior
Json selected using JSON path returns valid json that can be parsed using json parsing functions via processing pipelines.
## Current Behavior
Attempting to parse json returned produces an error:
```
Unable to parse JSON
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('r' (code 114)): was expecting double-quote to start field name
at [Source: (String)"{results=[{"gender":"female","name":{"title":"Mrs","first":"Margrit","last":"Stapf"},"location":{"street":{"number":726,"name":"Drosselweg"},"city":"Brandis","state":"Mecklenburg-Vorpommern","country":"Germany","postcode":22688,"coordinates":{"latitude":"-49.5036","longitude":"92.7654"},"timezone":{"offset":"+5:00","description":"Ekaterinburg, Islamabad, Karachi, Tashkent"}},"email":"margrit.stapf@example.com","login":{"uuid":"ad9b1327-232d-430b-9468-7d8cb653db69","username":"lazykoala841","pass"[truncated 698 chars]; line: 1, column: 3]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804) ~[graylog.jar:?]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:693) ~[graylog.jar:?]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:591) ~[graylog.jar
```
## Possible Solution
## Steps to Reproduce (for bugs)
1. Create a `JSON path from HTTP API` input
2. Set URL to `https://randomuser.me/api/`
3. Set json path to `$.results.0` (this can be validated as valid json path via https://jsonpath.com/ )
4. Observe that data in the result field is not valid json
```
{gender=male, name={title=Mr, first=Eeli, last=Wiita}, location={street={number=1553, name=Reijolankatu}, city=Oulu, state=Satakunta, country=Finland, postcode=10824, coordinates={latitude=-25.5239, longitude=-99.0959}, timezone={offset=+6:00, description=Almaty, Dhaka, Colombo}}, email=eeli.wiita@example.com, login={uuid=2fc2a9a0-10cc-4164-bcd6-36240d477367, username=ticklishgorilla304, password=scarlett, salt=gcWXeBtT, md5=a2b96a058776711a7e7a4c8567cee0f8, sha1=10afffcd61b2650762a265dd36ac460b53057aab, sha256=c745e81c83d6032d3cc11f2c8021b59741d27cd89ac1407f66ab9df87ce08d0b}, dob={date=1966-09-05T18:33:04.100Z, age=55}, registered={date=2005-06-15T20:27:05.149Z, age=17}, phone=02-824-424, cell=044-949-75-71, id={name=HETU, value=NaNNA677undefined}, picture={large=https://randomuser.me/api/portraits/men/31.jpg, medium=https://randomuser.me/api/portraits/med/men/31.jpg, thumbnail=https://randomuser.me/api/portraits/thumb/men/31.jpg}, nat=FI}
```
## Your Environment
* Graylog Version: Graylog 4.3.5+32fa802
* Java Version: 11.0.16
* Elasticsearch Version: Opensearch 1.3.4
* MongoDB Version: 4.4
* Operating System: Ubuntu Server 20.04 LTS
* Browser version: Chrome 104
Please let me know if there are any questions or additional information needed.
Contributor guide
Assessment
This issue has not been assessed yet.