Graylog2 / Graylog2/graylog2-server
Implementation of Percentage aggregation function not quite useful
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
5.2.x added the Percentage aggregation function, long hailed to be a future champion of the people
Unfortunately the current implementation isn't quite useful yet, chiefly because there is no way to demonstrate a limited subset of field values against the total count of messages with that field.
Example:
In Cloud, we monitor the load balancer traffic to customer setups (tenants). We examine the HTTP responses in these LB logs on a per tenant basis (and there are 40ish tenants). If say, 10%+ of HTTP responses for a particular tenant in the last 5 minutes are suddenly returning HTTP response codes of 40x/50x (failures) , that's a good indicator that something is wrong with that tenant.
What I want to produce is a single % value for each tenant, where 0% means all HTTP responses for that tenant are 20x or 30x, and plot those as per-tenant lines on a timechart.
There is no way to do that right now, which means we can't produce functional alerting with this aggregation function as-is.
This shortcoming isn't specific to this example, but to almost all use-cases of this function.
For example, the ability to show what % of logon attempts from a specific workstation failed requires the same ability to split the search by the column of [work station], include all results in the search (don't filter out logon success or failed from the search results), and define which value(s) to show the percentage of, against the total count of appearances of that field in messages.
For this to be possible we need to be able to define an array of values for a field, for the percentage to be counting against the total count of appearances of that field in messages. In this case, I would want to define that as where elb_status_code = 400, 401,402,403,404,405,406,407,500,501,502 etc etc etc.
It might look like this:

Which would result a single % value per tenant, showing what % of the total count of HTTP requests failed (returned one of the listed 40x or 50x responses)
Contributor guide
Assessment
This issue has not been assessed yet.