matrixorigin / matrixorigin/matrixone

[Bug]: distinct aggregates split vectors with SQL-equal signed-zero elements

Open
#28,861 0 comments 0 reactions 1 assignee Claimed by @XuPeng-SH View on GitHub
kind/bug needs-triage
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Description

Exact and approximate distinct aggregates count vector elements `+0.0` and `-0.0` as different values even though vector comparison and grouping treat the vectors as equal.

This affects both `VECF32` and `VECF64`.

## Environment

- MatrixOne: latest official `main`, commit `07fdd4ae0f80f287b93fc4b525fd296d5617abc7`
- Deployment: local standalone launch (isolated Log/TN/CN ports and data directory)

## Reproduction

```sql
drop database if exists vector_distinct_signed_zero_minimal;
create database vector_distinct_signed_zero_minimal;
use vector_distinct_signed_zero_minimal;

create table t(id int, v64 vecf64(2), v32 vecf32(2));
insert into t values
(1, '[0,1]', '[0,1]'),
(2, '[-0,1]', '[-0,1]');

select v64 = lead(v64) over(order by id),
v32 = lead(v32) over(order by id)
from t order by id limit 1;

select count(distinct v64), approx_count_distinct(v64),
hll_cardinality(hll_add_agg(v64)) from t;
select count(distinct v32), approx_count_distinct(v32),
hll_cardinality(hll_add_agg(v32)) from t;

select count(*) from (select v64 from t group by v64) g;
select count(*) from (select v32 from t group by v32) g;
```

## Actual behavior

```text
type equality COUNT DISTINCT APPROX DISTINCT HLL GROUP BY groups
VECF64 true 2 2 2 1
VECF32 true 2 2 2 1
```

The result reproduced identically in 3/3 fresh-table runs.

## Expected behavior

Each pair must contribute one distinct value, consistent with typed vector equality and `GROUP BY`.

## Controls

- scalar `FLOAT`/`DOUBLE` exact `COUNT(DISTINCT)` correctly returns one after #27434;
- scalar `FLOAT`/`DOUBLE` approximate/HLL paths remain affected and are tracked in #28856;
- vector `GROUP BY` correctly canonicalizes element signed zero and returns one group.

## Code-path analysis

`pkg/common/hashmap/keycodec` has canonical vector encoders for `VECF32`/`VECF64` which normalize signed-zero elements and are used by grouping. Exact distinct argument handling canonicalizes only scalar `FLOAT`/`DOUBLE`, while HLL hashes raw vector bytes. Both aggregate paths miss the typed vector canonicalizer.

## Suggested regression coverage

- signed zero in every vector position for `VECF32` and `VECF64`;
- exact DISTINCT, approximate DISTINCT, HLL add/merge, multi-column DISTINCT, DOP 1/>1, partial/final, and spill;
- non-zero, NULL, dimension, and multiple-distinct-value controls.

## Duplicate search

Open and closed issues were searched for vector signed zero, vector DISTINCT, HLL, raw-byte hashing, and #27434's scalar fix; no matching vector report was found.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.