Graylog2 / Graylog2/graylog2-server
Allow graylog to store empty strings to elasticsearch
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
We are transmitting json formatted log messages to graylog using nxog. On some point we have noted, that graylog stores different number of fields to the elasticsearch after they are parsed by corresponding pipeline. After short investigation we have found that missed fields are empty strings (like in_txt: "" ).
## Expected Behavior
Graylog sends empty strings to elasticsearch. By default custom log fields are not indexed, so it should be Ok from elasticsearch side. String (text) field type does not have null_value option, so its a question if empty string is counted as null value...
## Current Behavior
Graylog does not try to create field with empty string in elasticsearch.
## Possible Solution
Provide possibility to allow empty values. For example as checkbox in index configuration which allows to enable this feature to the custom indexes only in case dynamic templates does not allow creating empty strings. In this case customer will have to create custom index with related mapping to accept empty strings.
## Steps to Reproduce (for bugs)
1. create pipeline with rule to add field using function set_field("empty_string", "");
2. apply pipeline to the steam
3. send message to stream
4. check created fields
## Context
As part of our logging environment we are using [tlog](http://scribery.github.io/tlog/ ) to record user sessions to elasticsearch. There is a tlog-play player tool which allows to play recorded sessions directly from the elasticsearch, but it requires all required fields to be stored in elasticsearch index.
## Your Environment
* Graylog Version: 3.0.2-1
* Elasticsearch Version: 6.8.1
* MongoDB Version: 4.0.10
* Operating System: Debian 9.9
Contributor guide
Assessment
This issue has not been assessed yet.