matrixorigin / matrixorigin/matrixone
[Bug]: distinct aggregates split vectors with SQL-equal signed-zero elements
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Description
Exact and approximate distinct aggregates count vector elements `+0.0` and `-0.0` as different values even though vector comparison and grouping treat the vectors as equal.
This affects both `VECF32` and `VECF64`.
## Environment
- MatrixOne: latest official `main`, commit `07fdd4ae0f80f287b93fc4b525fd296d5617abc7`
- Deployment: local standalone launch (isolated Log/TN/CN ports and data directory)
## Reproduction
```sql
drop database if exists vector_distinct_signed_zero_minimal;
create database vector_distinct_signed_zero_minimal;
use vector_distinct_signed_zero_minimal;
create table t(id int, v64 vecf64(2), v32 vecf32(2));
insert into t values
(1, '[0,1]', '[0,1]'),
(2, '[-0,1]', '[-0,1]');
select v64 = lead(v64) over(order by id),
v32 = lead(v32) over(order by id)
from t order by id limit 1;
select count(distinct v64), approx_count_distinct(v64),
hll_cardinality(hll_add_agg(v64)) from t;
select count(distinct v32), approx_count_distinct(v32),
hll_cardinality(hll_add_agg(v32)) from t;
select count(*) from (select v64 from t group by v64) g;
select count(*) from (select v32 from t group by v32) g;
```
## Actual behavior
```text
type equality COUNT DISTINCT APPROX DISTINCT HLL GROUP BY groups
VECF64 true 2 2 2 1
VECF32 true 2 2 2 1
```
The result reproduced identically in 3/3 fresh-table runs.
## Expected behavior
Each pair must contribute one distinct value, consistent with typed vector equality and `GROUP BY`.
## Controls
- scalar `FLOAT`/`DOUBLE` exact `COUNT(DISTINCT)` correctly returns one after #27434;
- scalar `FLOAT`/`DOUBLE` approximate/HLL paths remain affected and are tracked in #28856;
- vector `GROUP BY` correctly canonicalizes element signed zero and returns one group.
## Code-path analysis
`pkg/common/hashmap/keycodec` has canonical vector encoders for `VECF32`/`VECF64` which normalize signed-zero elements and are used by grouping. Exact distinct argument handling canonicalizes only scalar `FLOAT`/`DOUBLE`, while HLL hashes raw vector bytes. Both aggregate paths miss the typed vector canonicalizer.
## Suggested regression coverage
- signed zero in every vector position for `VECF32` and `VECF64`;
- exact DISTINCT, approximate DISTINCT, HLL add/merge, multi-column DISTINCT, DOP 1/>1, partial/final, and spill;
- non-zero, NULL, dimension, and multiple-distinct-value controls.
## Duplicate search
Open and closed issues were searched for vector signed zero, vector DISTINCT, HLL, raw-byte hashing, and #27434's scalar fix; no matching vector report was found.
Contributor guide
Assessment
This issue has not been assessed yet.