Graylog2 / Graylog2/graylog2-server
http JSON input -> JSON extractor, difficult to extract structured data values
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
In reference to:
https://groups.google.com/d/msg/graylog2/js9dBgEHaDA/-jk09IHrAAAJ
## Expected Behavior
I've got a JSON HTTP API which returns several data items which are of interested to me, something like this:
``` json
{
"objs": { "devs": 100, "subs": 200 },
"dbstats": { "read": 300, "write": 400 }
}
```
I've defined an HTTP JSON _input_ in Graylog, and was hoping to then use the JSON extractor to map the values of `subs`, `devs`, `read`, `write` into properties of the collected messages (for monitoring, charts, etc.)
## Current Behavior
Let's focus on `subs` and `devs` for now.
The HTTP JSON input has its `JSON path of data to extract` set to `$.objs`.
And then I'm trying to use JSON extractor to break that apart into `devs` and `subs`.
But that does't work because the HTTP JSON input stores the data (in `result`) as some sort of Java map.toString(), not valid JSON, and then the JSON extractor can't work wit that:
Expected values in `result`:
`{ "devs": 100, "subs": 200 }`
Actual values in `result`:
`{ devs=100, subs=200 }`
## Possible Solution
1 - Change the HTTP JSON extractor so it can store its output (`result`) as JSON. Maybe a setting so it's optional, preserving the current behavior.
OR
2 - Add a way to configure JSON path -> value mapping right in the HTTP JSON extractor, allowing for multiple values.
So instead of the current (still not working, but maybe possible with solution #1):
JSON HTTP input -> calls the HTTP JSON API -> takes $.objs and stores as JSON -> JSON extractor takes this JSON apart and adds the values of `subs` and `devs` to each message --
-- it would be possible to do this:
JSON HTTP input -> calls the HTTP JSON API -> takes `$.objs.devs` and stores as `devs`, takes `$.objs.subs` and stores as `subs`, etc.
A JSON Extractor would not be needed then here.
## Context
Trying to collect structured typed values from an HTTP JSON API using a HTTP JSON input.
## Your Environment
- Graylog Version: 2.0.3
- Elasticsearch Version: 2.3.5
- MongoDB Version: 3.2.9
- Operating System: Debian Linux 8.5
- Browser version: Google Chrome 52
Contributor guide
Assessment
This issue has not been assessed yet.