Graylog2 / Graylog2/graylog2-server

Allow redirection of queries to ES coordination nodes

Open
#6,020 4 comments 0 reactions 0 assignees View on GitHub
elasticsearch feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

In a large ES cluster with heavy bulk writes, queries can cause instability as they are treated at parity with write queue requests on the node they happen to land on.

Best practice for ES includes sending queries to coordination nodes- non-data, non-master ES nodes which have free resources to distribute the query request to relevant nodes containing the index shard data within search scope.

![image](https://user-images.githubusercontent.com/568957/59702209-d2525100-91bc-11e9-9833-83fc74214250.png)

## Expected Behavior
Graylog server.conf should be able to define coordination nodes(or other node types as required). Queries sent from graylog UI should be sent to coordination nodes rather than general ES data nodes.

## Current Behavior
Queries are sent to the general ES data nodes (indexing nodes).

## Possible Solution
Add server.conf option to specify hosts to send queries to. Can be optional and will allow the graylog query interface to use this list of hard-set nodes as the ones to send queries to.

## Context
Random instability in query results during heavy ES cluster load.

* Graylog Version: 3.0.2
* Elasticsearch Version: 6.7.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.