Graylog2 / Graylog2/graylog2-server

New Pipeline Function: to_json

Open
#23,278 0 comments 0 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## What?

New Pipeline Function: to_json

## Why?

Currently Graylog does not provide sufficient tooling for handling multidimensional json.

A good example of where Graylog struggles with this is with lookup tables. Graylog is able to receive a valid json payload from a lookup table, but unable to manipulate the table as json.

The lookup table function returns an object which cannot be converted to usable json. While Graylog can set the [top level] contents using `to_map` and then `set_fields`, this does not allow properly saving nested fields.

Graylog also does not properly convert to a string when using `to_string`. While this does technically work, the returned string is invalid json.

`to_json` would be able to accept a multidimensional object and concert it to a string that contains valid json. From here Graylog's existing JSON functions are sufficient.

Happy to discuss and show an example of the problem.

The only workaround is to handle the payload outside of Graylog, for example, using a HTTP JSON lookup table to a custom app, such as a python flask app. This is not ideal as it has a high bar for entry and not accessible to many people.

Also open to other ideas about how to solve this problem such as having the lookup table function have a parameter that can help address this issue (such as configuring the output type, map, json, etc)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.