Graylog2 / Graylog2/graylog2-server

TCP Output Load Balancing

Open
#4,976 0 comments 0 reactions 0 assignees View on GitHub
feature triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

In our case we have Graylog instances on client site and then Graylog instances in the cloud for that client. The client site GL send the logs to the cloud for further processing of the logs. currently in the cloud we have single node GL instances, but are looking to cluster these environments due to higher performance requirements. When we send the logs via TCP + TLS (sensitive client date needs to be encrypted when going out over the internet), GL only creates one TCP connection even if there are no

## Expected Behaviour
The GELF TCP output should send data in chunks, with a new TCP connection being created each time. For example, if it sent 5 seconds of messages then sent the next five seconds via a new TCP connection it would allow the connection to be load balanced.

## Current Behaviour
The GELF TCP + TLS output creates one long TCP connection and therefore the load balancer just sends the connection to one node and waits for the next connection, so it can load balance it. But as the first GL instance won't initiate another TCP connection the load balancer just sends all the traffic to one node.

## Possible Solution
If the output were able to send logs for a few seconds then initiate a new TCP connection, it would allow for load balancing of a TCP output from Graylog to Graylog.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.