Graylog2 / Graylog2/graylog2-server
IPfix showing incorrect data transfer volumes when packet sampling is used.
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
On Ipfix inputs Graylog ignores OPTIONS data.
The result is, Graylog shows incorrect volumes of data transfers when sampling is used for IPfix. Sampling is used very often on high bandwidth environments.
## Expected Behavior
At least with Juniper routers, two different flow data packets are sent:
* Flow data (the typical packets with a set of flows per packet)
* Options data which contains information such as configuration (flow timeouts, sampling rate, etc)
The important data is, of course, in the flow data packets. There you find source and destination addresses, source and destination ports, protocols etc per flow.
**But** if, for instance, sampling is used, the octetDeltaCount and packetDeltacount fields do not reflect the volume of data transmitted. In typical configurations sample rates of 1:1000 are used (one packet every 1000), hence the actual data volume transmitted would be 1000 * octetDeltaCount, etc.
So, Graylog should correct those values (octetDeltaCount and packetDeltaCount) with the sample rate.
There are two ways to address this issue:
* Manual configuration
* Automatic detection
Manual configuration can be tricky because it would be enough to set up a sample rate configuration value for an input; an input can receive data from multiple collectors, so ideally it would require source:input tuples.
Automatic detection is not complicated. At least with Juniper routers, is based on a second kind of IPfix data sent: OPTIONS.
OPTIONS data is sent at longer intervals than flow data of course. In our case, every 10 minutes.
## Current Behavior
Graylog should show the proper quantities. It does not.
## Possible Solution
(**Note that this is not complete, it's just a sketch that would require deeper discussion**)
Juniper routers with the default templates send the following options fields:
* Flow exporter Id
* Packets exported
* Flows exported
* System init time
* Exporter IPv4 Address
* Exporter IPv6 address
* Sampling interval
* Flow active timeout
* Flow inactive timeout
* Export protocol version (10 for IPfix)
* Export transport protocol
I am going to describe a possible solution just for the sample rate value but if the data is kept historically it would be a good idea to add these fields to the flow information because parameters such as timeouts could be changed.
Henceforth, flow data is IPfix packets describing flows, options data is IPfix packets with the mentioned options.
I assume that right now the IPfix input must wait for a template in order to process flow data. Once the template has been received it is held in memory so that each flow data packet can be decoded. That template would be replaced by a new one if there are changes.
The same should be done with the options data. Before receiving an options template, options data should be ignored.
Once a template and options data is received, it would require:
* Keeping the options values in memory
* For each flow, do the following
* Multiply the octetDeltaCount and packetDeltaCount by the sampling rate
* Add fields with the values of at least: sampling interval, flow timeouts
Now, there might be some confusion with flow timestamps but I will address them on another issue.
That said, I would suggest adding a manual sample rate configuration to IPfix inputs. In that case documentation should recommend defining a dedicated input for each IPfix source or at least grouping sources with the same sample rate so that they go to an input with the proper manual sample rate configuration.
## Simple workaround
I am going to define a CSV table specifying the sample rate for each source, so that I can use a simple pipeline rule to correct the
## Steps to Reproduce (for bugs)
For example, I have done a test running iperf3 for 10 minutes with the bandwidth capped at 100 Mbps.
While nfdump/nfsen (a really old and wise Netflow monitoring package) which takes into account the configured sampling gives me a total of 15.8 GB transmitted, Graylog gives me 1:1000 of that quantity, 15.8 MB. Which is consistent with not applying the proper sampling.
## Context
I was trying to design some dashboards reflecting the data volumes exchanged over a network. While sampling reduces accuracy, in this case Graylog is certainly inferior to proper Netflow/IPfix monitoring software.
Right now I am using IPfix, not Netflow v9 but I can arrange a test with another router brand if it would be helpful.
## Your Environment
* Graylog Version: 7.0.3
* Java Version: OpenJDK 21
* OpenSearch Version: 2.19.4
* MongoDB Version: 7.0.28
* Operating System: FreeBSD 14.3
* Browser version: Safari 26.2
I have made sure that the system is *not* losing UDP packets due to buffer overruns.
The current uptime is 165 days and the number of UDP buffer overruns is 0.
As I mentioned on another issue pertaining socket buffer size I am running with 1 MB buffers. It required me to specify "0" as the input buffer size due to an improper handling of socket input buffer size on Linux.
The workaround is working fine on FreeBSD.
## Checklist
[] This issue fix need to be backported.
[NO] Does this issue have **security** implications?
Contributor guide
Assessment
This issue has not been assessed yet.