Graylog2 / Graylog2/graylog-plugin-integrations

Source field becomes incorrect when ingesting netflow data to graylog docker container

Open
#709 2 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Java
Stars
17
Forks
14
PR merge metrics
No merged PRs in 30d

Description

## Expected Behavior
Source field should be correct and show the IP of the Netflow source

## Current Behavior
TLDR: In graylog container, something triggers the netflow input to switch from storing the correct source field to storing the internal gateway IP for the docker network the container is in.

I'm running graylog, elasticsearch and mongo as containers from a docker-compose file more or less the same as that given in the documentation.
I have a Netflow input set up, override_source is not set. Into that I am feeding Netflow v9 records from two different systems, running different netflow probe software (a Linux box running netprobe and a BSD box running softflowd). All is well, and the data is correct. At some point, *something* happens, and the source field in the data changes from the IP of the netflow probe machine to the IP of the internal gateway of the docker network. This happens to data from both netflow probe machines.

I believe from a small amount of testing that the 'something' that triggers it is the restart of the docker container, whether by the host updating docker or stopping/starting the container. That certainly seemed to cause it to start to use the docker host's IP as the source rather than the actual IPs but there may be other triggers.
Once in that state, restarting the probe software on the remote boxes seems to be the only way to get it to start logging the correct source field again.

The issue does seem to be specific to the Netflow module, other inputs (beats, syslog) are working fine. I can't currently test whether there's any way to trigger the same issue on a non-container version.

## Context
It's problematic if I want to search based on which box sent netflow data, or to apply pipeline rules to normalize the data coming from the two sources if I relied on that obvious field to tell the sources apart.

## Your Environment

* Graylog Version: Graylog 4.0.1+6a0cc0b (docker container tag graylog/graylog:4.0)
* Java Version: 1.8.0_275
* Elasticsearch Version: 7.10.1 (docker container tag elasticsearch-oss:7.10.1)
* MongoDB Version: 3 (docker container tag mongo:3)
* Operating System: Ubuntu 18.04
* Browser version:
* Docker version: 20.10.2

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.