influxdata / influxdata/influxdb

movingMedian function

Open
#24,147 0 comments 2 reactions 0 assignees View on GitHub
area/flux kind/feature-request
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

__Proposal:__
Similar to movingAverage, there should be a function for movingMedian to filter out anomalous data points. There have been requests for such a function in multiple issues, but I still see no way to accomplish this.

EDIT: Or, even more flexible, a function like aggregateWindow, but that you specify a number of points rather than a time range. Such that `movingWindow(fn: median, n: 3)` would be the same as my proposed `movingMedian(n: 3)`

__Current behavior:__
You must window the data by time, rather than by number of data points to get a moving median.

__Desired behavior:__
A way to window data by number of points, or a movingMedian function to do the desired filtering directly.

__Alternatives considered:__
Use an `aggregateWindow(every: 15m, period: 45m, fn: median)` or something of that nature. This works depending on how regular your data points are, but it also leaves several windows at the beginning and end of the window which only had less than 3 points when the median function was applied.

__Use case:__
We have some sensors that occasionally emit a wrong data point which isn't in line with the rest of the values emitted by the sensor. Being able to grab the moving median of 3 points ensures we never do any calculations which use the spurious points.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing movingAverage and aggregateWindow functions mentioned in the issue, including how they handle point windows and incomplete windows. Compare the proposed movingMedian(n: 3) and movingWindow(fn: median, n: 3) behaviors, then define the supported API and verify that sensor data can be filtered by a fixed number of points.

Written by the indexing model from the issue text.

Assessment

Domain
data, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.