Graylog2 / Graylog2/graylog2-server

Provide ability to filter/search by nested fields in UI

Open
#5,904 0 comments 5 reactions 0 assignees View on GitHub
feature processing search triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## Expected Behavior
Currently we have a pipeline configured for a "labels" field in the message which parses the "labels" field in the incoming message using `parse_json()` function and creates a map of labels and stores them as nested fields under the parent field `labels`. Below is the mapping of the "labels" field in elasticsearch.
```
"labels" : {
"properties" : {
"app" : {
"type" : "keyword"
},
"chart" : {
"type" : "keyword"
},
"component" : {
"type" : "keyword"
},
"container" : {
"type" : "keyword"
},
"namespace" : {
"type" : "keyword"
},
"pod" : {
"type" : "keyword"
},
"pod-template-hash" : {
"type" : "keyword"
},
"release" : {
"type" : "keyword"
},
"tag" : {
"type" : "keyword"
}
}
}
```

The expected behavior is to be able to filter and search for messages using these nested fields.

## Current Behavior
Currently, the UI does not show nested fields. In this example, it only shows the "labels" field and does not show the nested labels.app, labels.pod etc. We have the same limitation with the query auto-complete feature, where it shows only the parent field, and not nested fields (note that the query works fine if we type out the nested field manually in the search query; it just doesn't auto complete)

![image](https://user-images.githubusercontent.com/16399189/56643455-4aa60580-6697-11e9-8810-23d0be907b2a.png)

![image](https://user-images.githubusercontent.com/16399189/56643471-509be680-6697-11e9-84e0-e547e187025e.png)

## Context
As we intend to have more than 15-20 labels, it will be difficult for users to search and filter using these fields as they have to remember the fields and enter it manually in the query each time. Having this feature would also help generate "Quick Values" and "Statistics" on these fields which can be very useful. One workaround is that we flatten the labels field as label_app, label_component, label_pod, but we will lose the structure of the message.

It would be really useful have a ui feature that allows easy searching, filtering and analysis using nested fields.

## Your Environment
* Graylog Version: 3.0.1
* Elasticsearch Version: 6.6.2
* MongoDB Version: 4.0.8
* Operating System: Ubuntu 16.04

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.