Improve performance in HyperLogLogDistinctAggregation
- Dominant language
- Java
- Stars
- 4.4k
- Forks
- 616
- Avg merge
- 14h 49m
- Merged PRs (30d)
- 206
Description
HyperLogLogDistinctAggregation contains the following code executed for each document that matches the top-level query:
```
BytesRef ref = super.values.lookupOrd(super.values.nextOrd());
byte[] bytes = NetworkUtils.formatIPBytes(ref).getBytes(StandardCharsets.UTF_8);
var hash = state.isAllOn4_1() ?
MurmurHash3.hash64(bytes, 0, bytes.length)
: MurmurHash3.hash128(bytes, 0, bytes.length, 0, super.hash128).h1;
state.addHash(hash);
```
For low-cardinality fields it will be more efficient to calculate the hashes of all values in the terms dictionary up-front, store these in an array, and use the per-document ordinal values to look up the pre-calculated hash, rather than doing a `lookupOrd` and re-calculating the hash on every doc.
Contributor guide
Assessment
This issue has not been assessed yet.