Graylog2 / Graylog2/graylog2-server

HTTP JSONPath requests with multiple keys

Open
#8,873 0 comments 0 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## HTTP JSONPath requests with multiple keys

Currently the `HTTP JSONPath` data adapter only supports searching for a single key, URL-encoding said key.

I propose Graylog supports multiple keys, optionally named, for this data adapter making the whole thing more useful and easier to work with.

Optionally the `lookup` function could accept a map or list of keys, for usage in pipelines.

## Why?

Many APIs implement path concatenation in their routes like this:

`GET /house/buckingham-palace/tenants`

`[
{
"name": "Elizabeth",
"title": "Queen"
},
{
"name": "Tommy",
"title": "Gardener"
}
]`

with an optional filter:

`GET /house/buckingham-palace/tenants?title=Gardener`
`[
{
"name": "Tommy",
"title": "Gardener"
}
]`

The case for multiple keys becomes obviously useful, limiting the size of the returned set and making the purpose immediately apparent.

Furthermore, Graylog currently URL-encodes the entire key, so the work-around of concatenating the path and filter in a single key is impossible since the slashes are encoded

When looking up values in which the "key" is a relation between values hierachicaly related to each other, from multiple fields, this would allow for a cleaner result.

for example using the name of a network devices and the name of an interface to lookup a description,

I work with an in-house API, which is capable of doing such a thing, for `my-device.my.domain` and `Ethernet1/10`:

`GET /devices/my-device.my.domain/ethernetInterface?filter=(name=Ethernet1%2F10)`

will work, however if I use a single key to build this URL, Graylog will encode every slash, mangling the route and making the API unusable, whereas if there were multiple keys, this would not be a problem.

loking up a key such as `my-device.my.domain/ethernetInterface?filter=(name=Ethernet1/10)` produces this relevant log line:

`2020-08-26T15:55:26.780+01:00 WARN [HTTPJSONPathDataAdapter] HTTP request for key failed: Response{protocol=http/1.1, code=400, message=, url=https://api.domain/redacted/path/device/my-device.my.domain%2FethernetInterface%3Ffilter%3D%28name%3DEthernet1%2F10%29}`

## Your Environment

* Graylog Version: 2.4
* Elasticsearch Version: 6.5
* MongoDB Version: 3.6.16
* Operating System: CentOS 7
* Browser version:

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.