Graylog2 / Graylog2/graylog2-server
Make graylog install custom index mappings into ES for standard field types
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
https://github.com/Graylog2/graylog2-server/issues/3603 looks like a similar problem that probably also could be avoided by (pre)setting the index format
## Expected Behavior
Don't allow Elasticsearch indices index format to be guessed into something that breaks future log processing
## Current Behavior
Then the first message is written to a new indices, Elasticsearch guesses the index format based on what comes in. In most cases it works well, but sometimes tricky logs can make elasticsearch guess the format wrong. Then you typically get messages like in server.log:
https://community.graylog.org/t/error-messages-failed-to-index-499-messages-please-check-the-index-error-log-in-your-web-interface-for-the-reason-error-one-or-more-of-the-items-in-the-bulk-request-failed-check-bulkresult-getitems-for-more-information/5304
## Possible Solution
For the syslog supplied fields [application_name, facility, level, process_id] it would be good if the field type could be enforced by adding a custom index mapping at (graylog) installation/initiation time.
## Steps to Reproduce (for bugs)
## Context
It would make the default graylog installation less error prune to syslog messages that are formatted in a sloppy way.
## Your Environment
* Graylog Version: 2.4.4
* Elasticsearch Version: 5.6.9
* MongoDB Version: 2.6.10
* Operating System: Ubuntu 16.04
Contributor guide
Assessment
This issue has not been assessed yet.