cockroachdb / cockroachdb/cockroach

sql: VARIANCE function returns negative value for large DECIMAL when all non-NULL values are equal, with error amplified under distributed aggregation

Open
#173,063 2 comments 0 reactions 1 assignee Claimed by @ZhouXing19 View on GitHub
branch-master C-bug O-community T-sql-queries X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

When all non‑NULL inputs to `VARIANCE(c0)` are the same huge DECIMAL value (`99999999999999999999.9999999999`), the function should return `0`.
Instead, CockroachDB v26.2.3 returns a large **negative** value:

- Single‑table query (local plan): **-1111111111.1111111111**
- Multi‑table JOIN query that produces the same data (full‑distribution plan): **-3333333333.3333333333** – exactly **3×** the single‑table result.

**To Reproduce**

**What did you do?**

Ran `VARIANCE()` on a DECIMAL column where every non‑NULL row holds the same huge value, first in a simple scan, then in an equivalent multi‑table JOIN that forces full distribution.
If possible, provide steps to reproduce the behavior:

**Steps to reproduce:**

1. Start a single‑node CockroachDB v26.2.3 cluster (insecure mode).
2. Connect with psql.
3. Run the following SQL:
```sql
DROP DATABASE IF EXISTS repro_cockroachdb82_db12_min CASCADE;
CREATE DATABASE repro_cockroachdb82_db12_min;
USE repro_cockroachdb82_db12_min;

SET vectorize = 'on';
SET distsql = 'always';

CREATE TABLE vp_source (
vp_rowid INT8 PRIMARY KEY,
c0 DECIMAL,
c1 INT4
);
CREATE INDEX src_c1_c0 ON vp_source (c1, c0);

CREATE TABLE vp_l (vp_rowid INT8 PRIMARY KEY);
CREATE TABLE vp_r (
vp_rowid INT8 PRIMARY KEY,
c0 DECIMAL,
c1 INT4
);

INSERT INTO vp_source VALUES
(1, NULL, 1), (2, NULL, 1),
(3, 99999999999999999999.9999999999, 1),
(4, 99999999999999999999.9999999999, 1),
(5, 99999999999999999999.9999999999, 1),
(6, 99999999999999999999.9999999999, 1),
(7, 99999999999999999999.9999999999, 1),
(8, 99999999999999999999.9999999999, 1),
(9, 99999999999999999999.9999999999, 1),
(10, 99999999999999999999.9999999999, 1),
(11, 99999999999999999999.9999999999, 1),
(12, 99999999999999999999.9999999999, 1);

INSERT INTO vp_l SELECT vp_rowid FROM vp_source;
INSERT INTO vp_r SELECT vp_rowid, c0, c1 FROM vp_source;

ALTER TABLE vp_source SPLIT AT VALUES (3), (5), (7), (9), (11);
ALTER TABLE vp_l SPLIT AT VALUES (3), (5), (7), (9), (11);
ALTER TABLE vp_r SPLIT AT VALUES (3), (5), (7), (9), (11);

ALTER TABLE vp_source SCATTER;
ALTER TABLE vp_l SCATTER;
ALTER TABLE vp_r SCATTER;

-- Single‑table query (local plan)
SELECT 'single' AS shape, VARIANCE(c0)
FROM vp_source@src_c1_c0;

-- Multi‑table query (full‑distribution plan)
SELECT 'string-pack' AS shape, VARIANCE(c0)
FROM vp_l l
JOIN (
SELECT vp_rowid, concat('vp:', vp_rowid::STRING) AS payload
FROM vp_r
) sp ON concat('vp:', l.vp_rowid::STRING) = sp.payload
JOIN vp_r r ON r.vp_rowid = sp.vp_rowid;
```
4. Observe the results:
```
shape | variance
----------+---------------------
single | -1111111111.1111111111
string-pack | -3333333333.3333333333
```
Both are negative; the true variance for this data set is `0` (all non‑NULL values are identical).
The multi‑table result is exactly **3×** the single‑table error.

**Expected behavior**
`VARIANCE(c0)` should return `0` (or a very small non‑negative number within precision limits).
It should never return a large negative number, and the result should be independent of the query plan shape (local vs. full‑distribution).

**Additional data / screenshots**
Execution plans:

**Single‑table (local):**
```
distribution: local
group (scalar)
scan vp_source@src_c1_c0
aggregate: variance(c0)
```

**Multi‑table (full):**
```
distribution: full
group (scalar)
hash join: column14 = payload
render
scan vp_l@vp_l_pkey
merge join: vp_rowid = vp_rowid
render
scan vp_r@vp_r_pkey
scan vp_r@vp_r_pkey
aggregate: variance(r.c0)
```

The negative values indicate catastrophic cancellation in `sum_sqr - (sum*sum)/count`.
The 3× amplification matches the number of ranges / merge tree branches interacting with the duplicate scan of `vp_r`.

**Environment:**
- **CockroachDB version:** CockroachDB CCL v26.2.3
- **Docker image:** cockroachdb/cockroach:v26.2.3
- **Server OS:** Ubuntu 20.04.6 LTS x86_64, Linux 5.4.0-204-generic
- **Client app:** psql (PostgreSQL connection, sslmode=disable)
- **Cluster mode:** single‑node, insecure. The test explicitly executed `SPLIT AT` and `SCATTER` to create multiple ranges and set `distsql = 'always'`. Since there is only one node, this does not result in cross‑node replica placement or cross‑node execution; however, the two‑phase DistSQL aggregation still occurred locally.

Jira issue: CRDB-66364

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.