Graylog2 / Graylog2/graylog2-server

Input type JSON path from HTTP API doesn't store valid json when specifying JSON path of data to extract

Open
#13,344 7 comments 0 reactions 0 assignees View on GitHub
bug to-verify triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

When using the input type `JSON path from HTTP API`, you are required to provide a value for `JSON path of data to extract`.

However, any value placed in this field other than `*` returns invalid json that cannot be parsed.

## Expected Behavior

Json selected using JSON path returns valid json that can be parsed using json parsing functions via processing pipelines.

## Current Behavior

Attempting to parse json returned produces an error:

```
Unable to parse JSON
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('r' (code 114)): was expecting double-quote to start field name
at [Source: (String)"{results=[{"gender":"female","name":{"title":"Mrs","first":"Margrit","last":"Stapf"},"location":{"street":{"number":726,"name":"Drosselweg"},"city":"Brandis","state":"Mecklenburg-Vorpommern","country":"Germany","postcode":22688,"coordinates":{"latitude":"-49.5036","longitude":"92.7654"},"timezone":{"offset":"+5:00","description":"Ekaterinburg, Islamabad, Karachi, Tashkent"}},"email":"margrit.stapf@example.com","login":{"uuid":"ad9b1327-232d-430b-9468-7d8cb653db69","username":"lazykoala841","pass"[truncated 698 chars]; line: 1, column: 3]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804) ~[graylog.jar:?]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:693) ~[graylog.jar:?]
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:591) ~[graylog.jar
```

## Possible Solution

## Steps to Reproduce (for bugs)

1. Create a `JSON path from HTTP API` input
2. Set URL to `https://randomuser.me/api/`
3. Set json path to `$.results.0` (this can be validated as valid json path via https://jsonpath.com/ )
4. Observe that data in the result field is not valid json

```
{gender=male, name={title=Mr, first=Eeli, last=Wiita}, location={street={number=1553, name=Reijolankatu}, city=Oulu, state=Satakunta, country=Finland, postcode=10824, coordinates={latitude=-25.5239, longitude=-99.0959}, timezone={offset=+6:00, description=Almaty, Dhaka, Colombo}}, email=eeli.wiita@example.com, login={uuid=2fc2a9a0-10cc-4164-bcd6-36240d477367, username=ticklishgorilla304, password=scarlett, salt=gcWXeBtT, md5=a2b96a058776711a7e7a4c8567cee0f8, sha1=10afffcd61b2650762a265dd36ac460b53057aab, sha256=c745e81c83d6032d3cc11f2c8021b59741d27cd89ac1407f66ab9df87ce08d0b}, dob={date=1966-09-05T18:33:04.100Z, age=55}, registered={date=2005-06-15T20:27:05.149Z, age=17}, phone=02-824-424, cell=044-949-75-71, id={name=HETU, value=NaNNA677undefined}, picture={large=https://randomuser.me/api/portraits/men/31.jpg, medium=https://randomuser.me/api/portraits/med/men/31.jpg, thumbnail=https://randomuser.me/api/portraits/thumb/men/31.jpg}, nat=FI}
```

## Your Environment

* Graylog Version: Graylog 4.3.5+32fa802
* Java Version: 11.0.16
* Elasticsearch Version: Opensearch 1.3.4
* MongoDB Version: 4.4
* Operating System: Ubuntu Server 20.04 LTS
* Browser version: Chrome 104

Please let me know if there are any questions or additional information needed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.