[Feature] Add Apache Datasketches HLL sketches aggregate function
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.
### Description
PR:see: https://github.com/apache/doris/pull/63143
An aggregate function is required to process user data containing Datasketches HLL sketches. In many data aggregation scenarios, users pre‑aggregate detailed data in Hive using the sketching techniques provided by Apache Datasketches, and then analyze the resulting sketches across various OLAP engines. Compared with the HLL union aggregate functions natively offered by these engines, there are two key diff to using Datasketches HLL sketches: firstly, the use cases differ; and secondly, HLL sketches can be used seamlessly across different engines—for example, simultaneously in ES, Doris, and ClickHouse. Such requirements are common in many production environments.
### Use case
```
CREATE TABLE test.test_table (
id INT,
data STRING
)
DUPLICATE KEY(`id`)
DISTRIBUTED BY HASH(`id`) BUCKETS 10
PROPERTIES (
"replication_num" = "1"
);
INSERT INTO test.test_table (id, data) VALUES (1, FROM_BASE64_BINARY('AgEHCAMIBwjL18IEK/L7BoYv+Q11gWYHgbxdBntl5gj8LUIK'));
INSERT INTO test.test_table (id, data) VALUES (2, FROM_BASE64_BINARY('AwEHCAUIAAkKAAAAgbxdBoYv+Q3L18IEwekXBdIWcwc0omEOdYFmB/wtQgp7ZeYIK/L7Bg=='));
INSERT INTO test.test_table (id, data) VALUES (3, FROM_BASE64_BINARY('AwEHCAUIAAkKAAAAIjvrBcS1nwfGGWoEyHokBO8t9wc1qTEENkcJB7hWqQxZf9QNnuSbGA=='));
select ds_cardinality(data) from test.test_table;
```
then doris should return result 20
### Related issues
https://github.com/apache/doris/issues/26416,
https://github.com/apache/doris/issues/56246,
etc..
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
Assessment
This issue has not been assessed yet.