apache / apache/echarts

implement aggregate transformer

Open
#15,306 8 comments 14 reactions 1 assignee Claimed by @100pah View on GitHub
en new-feature
Dominant language
TypeScript
Stars
67.3k
Forks
19.8k
Avg merge
11d 14h
Merged PRs (30d)
8

Description

At present the there is only a simplest aggregate transformer in [echarts-simple-transform/aggregate](https://github.com/100pah/echarts-simple-transform/blob/main/src/aggregate.ts). We should better implement a built-in aggregate transformer in echarts.

## About the API
Think of whether the API of [echarts-simple-transform/aggregate](https://github.com/100pah/echarts-simple-transform/blob/main/src/aggregate.ts) appropriate.

## About group by
In [echarts-simple-transform/aggregate](https://github.com/100pah/echarts-simple-transform/blob/main/src/aggregate.ts) only on dimension can be set in group by. Do we need to support multiple dimension group by?

## About Methods

Some `methods` of aggregate already roughly implemented in [echarts-simple-transform/aggregate](https://github.com/100pah/echarts-simple-transform/blob/main/src/aggregate.ts):
+ `sum`
+ `count`
+ `first`
+ `average`
+ `Q1`
+ `Q2`/`median`
+ `Q3`
+ `min`
+ `max`

We need to make sure they are implemented correctly.

Other methods we may implement:
+ `distinct`: The count of distinct dimension values.
+ `variance`: The sample variance of dimension values.
+ `variancep`: The population variance of dimension values.
+ `stdev`: The sample standard deviation of dimension values.
+ `stdevp`: The population standard deviation of dimension values.
+ `stderr`: The standard error of dimension values.
+ `product`: The product of dimension values.
+ `valid`: The count of dimension values that are not `null`, `undefined`, `''`, `NaN`, `'-'`, or determined by user defined validation function.
+ ... anything else useful?

Some references: [vega aggregate](https://vega.github.io/vega/docs/transforms/aggregate/).

## About math calculation precision

We all known that the floating precision issue (like `0.1 + 0.2`). It might bring trouble in some scenarios like:
+ The math result is needed to be displayed in label.
+ The sum should equal to 100%.
+ Multiple by 100, 1000, ...

Some cases:
```js
0.1 + 0.2 === 0.30000000000000004
2.3 - 0.3 === 1.9999999999999998
33643.2 - 15918.3 === 17724.899999999998
146.39 - 62.83 === 83.55999999999999
1.09 - 0.13 === 0.9600000000000001
0.1 * 0.2 === 0.020000000000000004
16.08 * 100 === 1607.9999999999998
146.39 * 100 === 14638.999999999998
9999999999999999 === 10000000000000000
-11.699999999999986 === -11.699999999999987
```

We should also take care the `toFixed bug` like:
```js
1.005.toFixed(2) === '1.00' // rather than '1.01'
0.1.toFixed(20) === '0.10000000000000000555'
```

**Solution:**
+ A. Do not care about it until some users required.
+ B. Try to resolve it:
+ For arithmetic: use [addSafe](https://github.com/apache/echarts/blob/5.1.2/src/util/number.ts#L286) we already have and implement `multipleSafe` in appropriate place.
+ For comparison of two numbers, use:
```js
const EPSILON = (function () { // Solve compatibility issues
return Number.EPSILON ? Number.EPSILON : Math.pow(2, -52);
})();
function compare(a, b){
return Math.abs(a - b) < EPSILON;
}
```

See: [IEEE 754](https://en.wikipedia.org/wiki/IEEE_floating_point), , , , ... (lots of related references)

Some library:
+
+ [mathjs](https://mathjs.org/)
```js
(math.add(math.bignumber(0.1),math.bignumber(0.2)))
```
+ [decimal.js](http://mikemcl.github.io/decimal.js/)
```js
x = new Decimal(0.1)
y = 0.2
console.log(x.plus(y).toString()
```
+ [bignumber.js](https://github.com/MikeMcl/bignumber.js/blob/master/bignumber.js)
+ [big.js](https://github.com/MikeMcl/big.js/)

## About big number

Do not support.

## Test

Could find some test cases in some other math libraries.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.