Result Option: Aggregation - LSP Integration
- Dominant language
- TypeScript
- Stars
- 117
- Forks
- 51
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 4
Description
Feature Description
NOTE: This issue will track the implementation of the Aggregation capabilities from the perspective of the integration of the UI elements with the LSP. This issue is responsible for handling the functionality that needs to occur when a user applies the aggregation window to the result options and the query contents need to be edited by the LSP to contain the aggregation window statement.
The associated UI element task is: https://github.com/influxdata/ui/issues/5759
Users sometimes query with large datasets that have a density too large for display in a table view (too many rows for display or enough memory to display) or in a graph view (too many data points to make each data point discernible).
In the old Data Explorer, this was solved by utilizing the aggregatewindow flux function on queries. By default, an aggregatewindow function would get added to each query, unless the user specifically went in and turned it off. But this led to many other issues/complications which resulted in not everyone appreciating the experience. Some of the issues:
1 - Users would not always get the data they expected when doing a query because, for instance, the average of a set of values was different than the actual values they were expecting to see, because they didn't understand about the auto-aggregation
2 - For datasets that contained fields that had string values instead of integers or floats, the query would error, because it was attempting to aggregate on those field values, which can't be done.
3 - Users didn't always want to aggregate their data and became very annoyed that they would immediately have to turn it off before they ran their query each time.
4 - The "custom" and "auto" options were confusing to people and they weren't always sure what they should set in order to turn on or off aggregations. Turning off aggregation meant switching to "custom" and then unchecking all of the boxes - which was not always intuitive. Sometimes users switched to the script editor just to delete the aggregationwindow function in the script to get around this.
With the new script editor, we want to create a better experience. We don't want to force auto-aggregation by default, but we need to give the user the opportunity to aggregate when working with large data sets, so we still need to give them options.
Objectives:
We want to provide them an easy, obvious and attractive way to turn on aggregations before running their query, but not force them to do so (opt-in).
We want to give them "?" help on what this functionality is to encourage them to use it and help explain why the would use it.
We want to eliminate or at least minimize the number of times the user would get a string aggregation error when using this functionality
General Acceptance Criteria
An option will be available in the UI for the user to enable aggregation after selecting a measurement, before the user submits the query.
If a way can be found to do it, the option would only be available to be enabled if the dataset doesn't have string data in its _value column ("last" query to get the data types after measurement selection), or would show a warning that string _value fields will be stripped from the result set.
If string values do appear in _columns then a warning will be shown along with the result set that string values were stripped from the output because of the aggregation
When aggregation is enabled, the user will be able to select the time window for the aggregation; the default time value will be 1 minute (it was 10s in old DE, need to visit this more)
The user will be able to type something for the window, or quick select from the following choices: 10 seconds, 20 seconds, 30 seconds, 1 minute, 10 minutes, 15 minutes, 30 minutes, 1 hour
The user will see an option to "fill missing values" with the same context help (this will be same as old DE)
The user will be able to select from the following aggregation options: mean, median, max, min, sum, stddev, first, last
The user will be able to select context help for each one of the aggregation options to see an explanation as to what they are (text TBD)
When aggregation options are made, and flux sync is on, the flux composition block will get updated with the aggregate window function containing their selections. When flux sync is turned off, the aggregate window line will get added to the end of the script
Notes
For best experience, we would need to find a way to examine the data type of the _value columns or filter any values; there may not be a good way to do that
In order to strip tables from the dataset with strings in the _value column, we may need underlying API updates (may not be possible from UI alone)
There is no "Auto" options being requested as existed with the old DE
The following options that were available in the old DE will not be available in this list: derivative, nonnegative derivative, distinct, count, increase, skew, spread, unique, sort as none of these options actually use the aggregatewindow function; we will look at adding some of these in another way besides the aggregation option
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.